Operating method for microphones and electronic device supporting the same

ABSTRACT

An electronic device which includes a plurality of microphones and an audio data processing module is provided. The plurality of microphones is operatively coupled to the electronic device, and the audio data processing module is capable of being implemented with at least one processor. The audio data processing module recognizes a specified command, based on first audio data collected using a portion of the plurality of microphones and executes a function or an application corresponding to second audio data collected using all the plurality of microphones, when the specified command is recognized.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation application of prior application Ser.No. 15/618,949, filed on Jun. 9, 2017, which was a continuationapplication of prior application Ser. No. 14/755,400, filed on Jun. 30,2015, which was issued as U.S. Pat. No. 9,679,563 on Jun. 13, 2017 andclaimed the benefit under 35 U.S.C. § 119(a) of a Korean patentapplication filed on Jun. 30, 2014 in the Korean Intellectual PropertyOffice and assigned Serial number 10-2014-0080540, the entire disclosureof which is hereby incorporated by reference.

TECHNICAL FIELD

The present disclosure relates to a method and a device capable ofoperating a plurality of microphones.

BACKGROUND

With the development of digital technologies, electronic devices whichperform communications and processing of personal information whilemoving may have launched in recent years. Such electronic devices may bedeveloped in the form of mobile convergence.

An electronic device may include a microphone to collect audio data. Theelectronic device may activate the microphone to collect audio data. Theelectronic device may store the collected audio data or may transmit itto other electronic device.

The above-described electronic device of the related art may include onemicrophone. For this reason, data collected through one microphone maybe information including a lot of noise. Accordingly, the electronicdevice of the related art may have a disadvantage in that accuracy ofvoice recognition of the collected audio data decreases.

The above information is presented as background information only toassist with an understanding of the present disclosure. No determinationhas been made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the present disclosure.

SUMMARY

Embodiments of the present disclosure are to address at least theabove-mentioned problems and/or disadvantages and to provide at leastthe advantages described below. Accordingly, an Embodiment of thepresent disclosure is to provide a microphone operating method capableof recognizing voice more accurately using a plurality of microphonesand an electronic device supporting the same.

Another Embodiment of the present disclosure is to provide a microphoneoperating method and an electronic device supporting the same, capableof utilizing at least one of a plurality of microphones above all andoperates the plurality of microphones according to a condition, therebymaking it possible to use power efficiently.

In accordance with an Embodiment of the present disclosure, anelectronic device is provided. The electronic device includes aplurality of microphones operatively coupled to the electronic deviceand an audio data processing module capable of being implemented with atleast one processor. The audio data processing module is configured torecognize a specified command, based on first audio data collected usinga portion of the plurality of microphones and to execute a function oran application corresponding to second audio data collected using allthe plurality of microphones, when the specified command is recognized.

In accordance with another Embodiment of the present disclosure, amicrophone operating method is provided. The method includes collectingfirst audio data using a portion of a plurality of microphonesoperatively coupled to an electronic device, recognizing a specifiedcommand, based on the first audio data, and executing a function or anapplication, corresponding to second audio data collected using all ofthe plurality of microphones, based on recognition of the specifiedcommand.

Other Embodiments, advantages, and salient features of the disclosurewill become apparent to those skilled in the art from the followingdetailed description, which, taken in conjunction with the annexeddrawings, discloses various embodiments of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other Embodiments, features, and advantages of certainembodiments of the present disclosure will be more apparent from thefollowing description taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 illustrates an electronic device operation environment includinga plurality of microphones according to an embodiment of the presentdisclosure;

FIG. 2 illustrates an electronic device which operates microphones,based on an audio codec and an audio data processing module, accordingto an embodiment of the present disclosure;

FIG. 3 illustrates an electronic device which uses microphones based ona low-power processing module and an audio data processing module,according to an embodiment of the present disclosure;

FIG. 4 illustrates an electronic device which uses microphones based ona low-power processing module, an audio data processing module, and anaudio codec according to an embodiment of the present disclosure;

FIG. 5 illustrates an electronic device which supports microphoneintegrated employment based on a low-power processing module and anaudio codec, according to an embodiment of the present disclosure;

FIG. 6 illustrates an electronic device which supports low-powermicrophone integrated use based on a low-power processing module and anaudio codec, according to an embodiment of the present disclosure;

FIG. 7 illustrates a microphone operating method according to anembodiment of the present disclosure;

FIG. 8 illustrates a screen interface of an electronic device accordingto an embodiment of the present disclosure; and

FIG. 9 illustrates a hardware configuration of an electronic deviceaccording to an embodiment of the present disclosure.

Throughout the drawings, it should be noted that like reference numbersare used to depict the same or similar elements, features, andstructures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the present disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding, but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thepresent disclosure. In addition, descriptions of well-known functionsand constructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but are merely used by theinventor to enable a clear and consistent understanding of the presentdisclosure. Accordingly, it should be apparent to those skilled in theart that the following description of various embodiments of the presentdisclosure is provided for illustration purposes only and not for thepurpose of limiting the present disclosure as defined by the appendedclaims and their equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

The term “include,” “comprise,” “including,” or “comprising” used hereinindicates disclosed functions, operations, or existence of elements butdoes not exclude other functions, operations or elements. It should befurther understood that the term “include”, “comprise”, “have”,“including”, “comprising”, or “having” used herein specifies thepresence of stated features, integers, operations, elements, components,or combinations thereof but does not preclude the presence or additionof one or more other features, integers, operations, elements,components, or combinations thereof.

The meaning of the term “or” or “at least one of A and/or B” used hereinincludes any combination of words listed together with the term. Forexample, the expression “A or B” or “at least one of A and/or B” mayindicate A, B, or both A and B.

Terms such as “first”, “second”, and the like used herein may refer tovarious elements of various embodiments of the present disclosure, butdo not limit the elements. For example, such terms do not limit theorder and/or priority of the elements. Furthermore, such terms may beused to distinguish one element from another element. For example, “afirst user device” and “a second user device” indicate different userdevices. Without departing from the scope of the present disclosure, afirst element may be referred to as a second element, and similarly, asecond element may be referred to as a first element.

In the description below, when one part (or element, device, etc.) isreferred to as being “connected” to another part (or element, device,etc.), it should be understood that the former can be “directlyconnected” to the latter, or “electrically connected” to the latter viaan intervening part (or element, device, etc.). It will be furtherunderstood that when one component is referred to as being “directlyconnected” or “directly linked” to another component, it means that nointervening component is present.

Terms used in this specification are used to describe variousembodiments of the present disclosure and are not intended to limit thescope of the present disclosure. The terms of a singular form mayinclude plural forms unless otherwise specified.

Unless otherwise defined herein, all the terms used herein, whichinclude technical or scientific terms, may have the same meaning that isgenerally understood by a person skilled in the art. It will be furtherunderstood that terms, which are defined in a dictionary and commonlyused, should also be interpreted as is customary in the relevant relatedart and not in an idealized or overly formal sense unless expressly sodefined herein in various embodiments of the present disclosure.

Electronic devices according to various embodiments of the presentdisclosure may include a metal case. For example, the electronic devicesmay include at least one of smartphones, tablet personal computers(PCs), mobile phones, video telephones, electronic book readers, desktopPCs, laptop PCs, netbook computers, personal digital assistants (PDAs),portable multimedia players (PMPs), Motion Picture Experts Group (MPEG-1or MPEG-2) Audio Layer 3 (MP3) players, mobile medical devices, cameras,wearable devices (e.g., head-mounted-devices (HMDs), such as electronicglasses), an electronic apparel, electronic bracelets, electronicnecklaces, electronic appcessories, electronic tattoos, smart watches,and the like.

According to various embodiments of the present disclosure, theelectronic devices may be smart home appliances including metal cases.The smart home appliances may include at least one of, for example,televisions (TVs), digital versatile disc (DVD) players, audios,refrigerators, air conditioners, cleaners, ovens, microwave ovens,washing machines, air cleaners, set-top boxes, TV boxes (e.g., SamsungHomeSync™, Apple TV™, or Google TV™), game consoles, electronicdictionaries, electronic keys, camcorders, electronic picture frames,and the like.

According to various embodiments of the present disclosure, theelectronic devices may include at least one of medical devices (e.g., amagnetic resonance angiography (MRA), a magnetic resonance imaging(MRI), a computed tomography (CT), scanners, and ultrasonic devices),navigation devices, GPS receivers, event data recorders (EDRs), flightdata recorders (FDRs), vehicle infotainment devices, electronicequipment for vessels (e.g., navigation systems and gyrocompasses),avionics, security devices, head units for vehicles, industrial or homerobots, automatic teller's machines (ATMs), and points of sales (POSs)including metal cases.

According to various embodiments of the present disclosure, theelectronic devices may include at least one of parts of furniture orbuildings/structures having communication functions, electronic boards,electronic signature receiving devices, projectors, and measuringinstruments (e.g., water meters, electricity meters, gas meters, andwave meters) including metal cases. The electronic devices according tovarious embodiments of the present disclosure may be one or morecombinations of the above-mentioned devices. Furthermore, the electronicdevices according to various embodiments of the present disclosure maybe flexible devices. It would be obvious to those skilled in the artthat the electronic devices according to various embodiments of thepresent disclosure are not limited to the above-mentioned devices.

Hereinafter, electronic devices according to various embodiments of thepresent disclosure will be described with reference to the accompanyingdrawings. The term “user” used herein may refer to a person who uses anelectronic device or may refer to a device (e.g., an artificialelectronic device) that uses an electronic device.

FIG. 1 illustrates an operation environment of an electronic deviceincluding a plurality of microphones according to various embodiments ofthe present disclosure.

Referring to FIG. 1, an electronic device operation environment maycontain an electronic device 100, an electronic device 102, anelectronic device 104, a network 162, and a server device 106. In theelectronic device operation environment, the electronic device 100 maysupport voice recognition of received audio data and a function processaccording to the voice recognition. The electronic device 100 mayinclude a plurality of microphones and may allow at least one of theplurality of microphones to maintain an active state. The electronicdevice 100 may allow the reform a program instruction (e.g., aprogramming module): Read Only Memory of the remaining microphones intoactive states, based on a result of analyzing audio data that the atleast one microphone collects.

The electronic device 102 may output audio data using a speaker and thelike. Audio data output from the electronic device 102 may be providedas an input of at least one of the plurality of microphones of theelectronic device 100. According to various embodiments of the presentdisclosure, the electronic device 102 may receive a result according tothe function process of the electronic device 100 or may perform afunction in conjunction with the electronic device 100. For example, inthe case where the electronic device 100 performs a function accordingto an analysis of specific audio data, the electronic device 102 mayform a communication channel with the electronic device 100.

The electronic device 104 may form a communication channel with theelectronic device 100 through the network 162. The electronic device 104may receive a result of the function process according to an analysis ofaudio data of the electronic device 100. For example, in the case wherethe electronic device 100 performs a call function according to theanalysis of audio data, the electronic device 104 may form acommunication channel in response to a request of the electronic device100.

The server device 106 may form a communication channel with theelectronic device 100 through the network 162. The server device 106 mayprovide information associated with voice recognition to the electronicdevice 100. According to various embodiments of the present disclosure,the server device 106 may provide service information associated with aspecific function performed at the electronic device 100 in response toa result of analyzing audio data. For example, the server device 106 mayprovide the electronic device 100 with a service page or content (e.g.,an audio file, an image file, a text file, and the like) associated withthe function process of the electronic device 100.

The network 162 may form a communication channel between the electronicdevices 100 and 104 or between the electronic device 100 and the serverdevice 106. The network 162 may transmit a variety of informationassociated with a function process of the electronic device 100.

Referring to FIG. 1, the electronic device 100 may include acommunication interface 110, an input/output interface 120, an audiocodec 130, a display 140, a memory 150, a processor 160, a low-powerprocessing module 170, an audio data processing module 180, and a bus190.

The electronic device 100 may include a microphone module MIC having atleast one microphone (Mic1 to MicN). The microphone module MIC mayoperate in response to a control of the audio data processing module180. According to an embodiment of the present disclosure, in the casewhere a detail recognition function of a voice recognition function isrequested, the electronic device 100 may activate the plurality ofmicrophones Mic1 to MicN to perform a voice recognition functionassociated with the detail recognition function. According to anembodiment of the present disclosure, in the case where a power savingfunction of the voice recognition function is requested, the electronicdevice 100 may activate one microphone, and when specific audio data iscollected, the electronic device 100 may perform the voice recognitionfunction to which the power saving function is applied, based on theplurality of microphones Mic1 to MicN.

The communication interface 110 may convey communications between theelectronic device 100 and an external device (e.g., the electronicdevice 104 or the server device 106). For example, the communicationinterface 110 may be coupled with the network 162 through wirelesscommunication or wired communication to communicate with the externaldevice. The wireless communication may include, for example, at leastone of Wi-Fi, Bluetooth (BT), near field communication (NFC), GPS, orcellular communication (e.g., long term evolution (LTE), LTE-advanced(LTE-A), code division multiple access (CDMA), wide code divisionmultiple access (WCDMA), universal mobile telecommunications service(UMTS), wireless broadband (WiBro) or global system for mobilecommunications (GSM)). The wired communication may include, for example,at least one of a universal serial bus (USB), a high definitionmultimedia interface (HDMI), recommended standard 232 (RS-232), or aplain old telephone service (POTS).

According to an embodiment of the present disclosure, the network 162may be a telecommunications network. The telecommunications network mayinclude at least one of a computer network, an internet, an internet ofthings, or a telephone network. According to an embodiment of thepresent disclosure, a protocol (e.g., a transport layer protocol, a datalink layer protocol or a physical layer protocol) for communicationsbetween the electronic device 100 and an external device may besupported by at least one of an application 154, an applicationprogramming interface 153, a middleware 152, a kernel 151, or thecommunication interface 110.

The communication interface 110 may include at least one communicationunit associated with a call function of the electronic device 100. Forexample, the communication interface 110 may include a variety ofcommunication units such as a mobile communication unit, a broadcastingreceiving unit such as a digital multimedia broadcasting (DMB) module ora digital video broadcasting-handheld (DVB-H) module, a near fieldcommunication unit such as a ZigBee module as a BT module or a NFCmodule, a Wi-Fi communication module, and the like. According to anembodiment of the present disclosure, the communication interface 110may form a communication channel associated with a voice call function,a video call function, and the like. The electronic device 100 mayactivate the voice recognition function while performing a call functionof the communication interface 110.

According to various embodiments of the present disclosure, thecommunication interface 110 may receive streaming data including audiodata, based on the Wi-Fi communication unit. The audio data processingmodule 180 may support a voice recognition function of streaming datareceived at a state where a communication channel is formed based on theWi-Fi communication unit. According to an embodiment of the presentdisclosure, the audio data processing module 180 may control functions,such as changing the Wi-Fi communication channel of the communication100, releasing the Wi-Fi communication channel of the communication 100,and the like, according to voice recognition. For example, if audio datasuch as “Hi Samsung, Stop streaming” is collected, the audio dataprocessing module 180 may recognize “Hi Samsung” as specific audio dataand “Stop streaming” as a function execution instruction. Accordingly,the communication interface 110 may stop a streaming data receivingfunction or may release a relevant communication channel.

According to various embodiments of the present disclosure, thecommunication interface 110 may form a communication channel with avoice recognition server device. For example, the communicationinterface 110 may transmit audio data, which is received aftercollecting specific audio data, to a specific voice recognition serverdevice according to a control of the audio data processing module 180.The communication interface 110 may receive a voice recognition resultfrom the specific voice recognition server device and may transfer thereceived voice recognition result to the audio data processing module180.

The input/output interface 120 may send an instruction or data receivedfrom a user through an input/output device (e.g., a sensor, a keyboard,or a touch screen) to the processor 160, the memory 150, thecommunication interface 110, or the audio data processing module 180,for example, through the bus 190. For example, the input/outputinterface 120 may provide the processor 160 with data associated with auser touch input through a touch screen. Furthermore, the input/outputinterface 120 may output an instruction or data, which is received fromthe processor 160, the memory 150, the communication interface 110, orthe audio data processing module 180, for example, through the bus 190,through the input/output device (e.g., a speaker or a display). Forexample, the input/output interface 120 may output voice data processedthrough the processor 160 to a user through a speaker.

The input/output interface 120 may generate an input signal of theelectronic device 100. The input/output interface 120 may include, forexample, at least one of a key pad, a dome switch, a touch pad(capacitive/resistive), a jog wheel, or a jog switch. The input/outputinterface 120 may be implemented in the form of button at the outside ofthe electronic device 100, and some buttons may be implemented withvirtual key buttons. According to an embodiment of the presentdisclosure, the input/output interface 120 may include a plurality ofkeys used to receive number or character information and to set avariety of functions. Such keys may include a menu call key, a screenon/off key, a power on/off key, a volume adjustment key, a home key, andthe like.

According to an embodiment of the present disclosure, the input/outputinterface 120 may generate an input event associated with activation ofa voice recognition function, an input event associated with selectionof a power saving function or a detail recognition function of the voicerecognition function, an input event associated with releasing (orinactivation) of the voice recognition function, and the like. Theinput/output interface 120 may further generate an input eventassociated with a control of a function executed according to the voicerecognition function, an event associated with an end of the executedfunction, and the like. The input event thus generated may be providedto the audio data processing module 180 so as to be applied to aninstruction or an instruction set associated with a control of arelevant function.

The audio codec 130 may process an audio signal of the electronic device100. For example, the audio codec 130 may send an audio signal receivedfrom the audio data processing module 180 to a speaker SPK. The audiocodec 130 may process an audio signal (e.g., voice and the like)received from at least one microphone and may send the processing resultto the audio data processing module 180. The audio codec 130 may convertan audio signal, such as voice and the like received from a microphoneinto a digital signal and may transfer the digital signal to the audiodata processing module 180. The audio codec 130 can be implemented witha chip independent of the audio data processing module 180.

According to an embodiment of the present disclosure, when the voicerecognition function is activated, the audio codec 130 may activate afirst microphone Mic1 to monitor collecting of specific audio data. Ifthe specific audio data is collected, the audio codec 130 may control toactivate the microphones Mic2 to MicN and may perform the detailrecognition function. The audio codec 130 may transfer a resultprocessed according to the detail recognition function to the audio dataprocessing module 180.

According to an embodiment of the present disclosure, when the voicerecognition function is activated, the audio codec 130 may activate theplurality of microphones Mic1 to MicN included in the microphone moduleMIC to control collecting of audio data. In this operation, if specificaudio data is collected using the plurality of microphones Mic1 to MicN,the audio codec 130 may perform at least a portion of a multi-microphonecontrol process, based on the collected audio data. The multi-microphonecontrol process may include at least one of a direction of arrivaldetermining function, a beamforming function, or a noise suppressionfunction. The audio codec 130 may transfer a result according to themulti-microphone control process to the audio data processing module180. Alternatively, the audio codec 130 may perform the voicerecognition function, based on a result according to themulti-microphone control process.

The display 140 may output a variety of screens corresponding tofunctions processed at the electronic device 100. For example, thedisplay 140 may output a waiting screen, a menu screen, a lock screen,and the like. According to an embodiment of the present disclosure, thedisplay 140 may output an icon or menu item associated with activationof the voice recognition function. The display 140 may output a screenassociated with a setting change of the voice recognition function. Thedisplay 140 may output information associated with a voice preprocessingfunction being executed, such as information associated with either apower saving function state or a detail recognition function state. Thedisplay 140 may output text information of audio data recognized inexecuting the voice recognition function, information found with respectto the text information, or an executed function screen. In the casewhere an error is generated in recognizing audio data, the display 140may output information of generation of an error. For example, in thecase where voice is not accurately recognized, the display 140 mayoutput an error message corresponding thereto.

According to an embodiment of the present disclosure, when the voicerecognition function is activated, the display 140 may outputinformation to one side of the display 140 indicating a position of atleast one microphone in the microphone module MIC. For example, thedisplay 140 may output information indicating a position of a firstmicrophone Mic1 in performing the voice recognition function that isbased on the first microphone Mic1. The display 140 may outputinformation indicating positions of the plurality of microphones Mic1 toMicN in performing the voice recognition function that is based on theplurality of microphones Mic1 to MicN.

The display 140 may display a screen in a landscape mode, in a portraitmode, and a screen change according to a change between the landscapemode and the portrait mode, based on screen/device orientation of theelectronic device 100. The display 140 may output information indicatinga position of at least one microphone according to each mode inexecuting the voice recognition function at a state where a mode of theelectronic device 100 is changed into the landscape mode or the portraitmode. Alternatively, the display 140 may output guide information forguiding so as to arrange in a landscape mode state or a portrait modestate in executing the voice recognition function. Outputting of theposition information and guide information of the microphone module MICcan be omitted according to user setting and the like.

The display 140 may include at least one of a liquid crystal display(LCD), a thin film transistor-LCD (TFT-LCD), a light emitting diode(LED), an organic LED (OLED), an active matrix OLED (AMOLED), a flexibledisplay, a bended display, and a 3D display. Some of the displays may beimplemented with a transparent display of a transparent type or a phototransparent type so as to view its outside.

Furthermore, the display 140 may be provided as a touch screen and maybe used as an input device as well as an output device. The display 140may be implemented to convert a variation in pressure forced to aspecific portion of the display 140, a variation in capacitanceoccurring at the specific portion of the display 140, or the like, intoan electrical input signal. The display 140 may be configured to detect(or sense) touch pressure as well as touched position and area.

The display 140 may be configured to include a touch panel and a displaypanel. The touch panel may be placed on the display unit. The touchpanel may be implemented in an add-on type where the touch panel isplaced on the display panel or an on-cell type or an in-cell type whereit is inserted in the display panel. The touch panel may provide theaudio data processing module 180 with a user input responding to a usergesture of the display 140. A user input generated by a touch means suchas a finger, a touch pen, or the like may include touch, multi-touch,tap, double tap, long tap, tap and touch, drag, flick, press, pinch in,pinch output, and the like.

The above-described user input may be defined with regard to the voicerecognition function. For example, the user input may be defined by aninput event for changing the power saving function or the detailrecognition function. Furthermore, the user input may be defined by aninput event for determining whether to use at least one selected fromthe plurality of microphones Mic1 to MicN included in the microphonemodule MIC, as a default microphone. The default microphone may be amicrophone that is activated above all (or always or periodically) tocollect specific audio data.

The memory 150 may store instructions or data received from theprocessor 160 or other components (e.g., the communication interface110, the input/output interface 120, the display 140, the audio dataprocessing module 180, and the like) or generated by the processor 160or the other components. The memory 150 may include, for example,programming modules such as a kernel 151, a middleware 152, anapplication processing interface (API) 153, and an application 154. Eachof the above-described programming modules may be implemented in theform of software, firmware, hardware, or a combination of at least twothereof.

The kernel 151 may control or manage system resources (e.g., the memory150, the processor 160, the bus 190, and the like) that are used toexecute operations or functions of remaining other programming modules,for example, the middleware 152, the API 153, or the application 154.Furthermore, the kernel 151 may provide an interface that accessesdiscrete components of the electronic device 100 on the middleware 152,the API 153, or the application 154 to control or manage them.

The middleware 152 may perform a mediation role so that the API 153 orthe application 154 communicates with the kernel 151 to exchange data.Furthermore, with regard to task requests received from the application154, for example, the middleware 152 may control (e.g., scheduling orload balancing) a task request using a method of assigning the priority,which enables the use of a system resource (e.g., the memory 150, theprocessor 160, the bus 190, or the like) of the electronic device 100,to at least one of the application 154.

The API 153 may be an interface through which the application 154controls a function provided by the kernel 151 or the middleware 152,and may include, for example, at least one interface or function (e.g.,an instruction) for a file control, a window control, image processing,a character control, or the like.

According to various embodiments of the present disclosure, theapplication 154 may include a short messaging service/multimediamessaging service (SMS/MMS) application, an e-mail application, acalendar application, an alarm application, a health care application(e.g., an application for measuring an exercise amount, a blood sugar,or the like), an environment information application (e.g., anapplication for providing air pressure, humidity, temperatureinformation, or the like) or the like. Additionally or generally, theapplication 154 may be an application associated with informationexchange between the electronic device 100 and an external electronicdevice (e.g., an electronic device 104). The application associated withinformation exchange may include, for example, a notification relayapplication for transmitting specific information to an externalelectronic device or a device management application for managing anexternal electronic device.

The notification relay application may include a function for providingan external electronic device (e.g., an electronic device 104) withnotification information generated from another application (e.g., amessage application, an e-mail application, a health care application,an environment information application or the like) of the electronicdevice 100. Additionally or generally, the notification relayapplication may receive, for example, notification information from anexternal electronic device (e.g., an electronic device 104) and mayprovide the notification information to a user. Additionally orgenerally, the notification relay application may manage (e.g., install,delete, or update), for example, the function (e.g., turn on/turn off ofan external electronic device itself, or a portion thereof, or controlof brightness or resolution of a screen) of at least a portion of theexternal electronic device (e.g., an electronic device 104)communicating with the electronic device 100, an application operatingon the external electronic device, or a service (e.g., a communication,or telephone, service or a message service) provided by the externalelectronic device.

According to various embodiments, the application 154 may include anapplication that is designated according to an attribute (e.g., the kindof electronic device) of the external electronic device (e.g., anelectronic device 104). For example, in the case where the externalelectronic device is an MP3 player, the application 154 may include anapplication associated with music reproduction. Similarly, in the casewhere the external electronic device is a mobile medical device, theapplication 154 may include an application associated with a healthcare. According to an embodiment of the present disclosure, theapplication 154 may include at least one of an application designated tothe electronic device 100 or an application received from the externalelectronic device (e.g., an electronic device 104 or a server 106).

According to various embodiments of the present disclosure, the memory150 may store a variety of programs and data associated with processingand controlling of data that relates to an operation of the electronicdevice 100. For example, the memory 150 may store an operating systemand the like. According to an embodiment of the present disclosure, thememory 150 may store a program associated with the voice recognitionfunction. The program associated with the voice recognition function mayinclude at least one of an instruction set used to register specificaudio data as specific audio data, an instruction set for comparingcollected audio data and the specific audio data, or an instruction setfor performing the voice recognition function according to the detailrecognition function when the specific audio data is collected. Theprogram associated with the voice recognition function may include aninstruction set (or at least one functions) associated with selection ofthe power saving function or the detail recognition function and aninstruction set for selecting a default microphone of the plurality ofmicrophones Mic1 to MicN in the power saving function. The programassociated with the voice recognition function may include aninstruction set for applying at least one process of a multi-microphoneprocess associated with the plurality of microphones Mic1 to MicN, aninstruction set for recognizing audio data collected according to themulti-microphone process, and an instruction set for executing aspecific function according to voice recognition.

According to an embodiment of the present disclosure, the memory 150 maystore a first voice recognition model 51 and a second voice recognitionmodel 53. The first voice recognition model 51 may be a voicerecognition model associated with specific audio data. For example, thefirst voice recognition model 51 may include audio data (e.g., specificaudio data or voice signal or a trained statistical model and errorrange information associated with a reference of the trained statisticalmodel) corresponding to a wake-up command for activating the voicerecognition function.

The first voice recognition model 51 may include utterance characterinformation of utterance for a specific isolated character and speakerclassification information associated with personal classification ofthe utterance for a specific isolated character. According to anembodiment of the present disclosure, the utterance characterinformation of the first voice recognition model 51 may be provided to adevice component which performs voice recognition of specific audio datain performing the voice recognition function that is based on the powersaving function. According to an embodiment of the present disclosure,the speaker classification information of the first voice recognitionmodel 51 may be provided to a device component which performs voicerecognition of specific audio data in performing the voice recognitionfunction that is based on the detail recognition function. Based on theabove-described condition, the electronic device 100 may classify voicesof specific persons and may execute a function according to a voiceinput from a relevant person. For example, if first audio datacorresponding to “Hi Samsung” is collected, the electronic device 100may determine whether the first audio data corresponds to voice signalsof specific persons, using the first voice recognition model 51. Ifdetermined as being a voice signal of a specific person, the electronicdevice 100 may perform voice recognition of second audio data laterreceived, for example, “Oh Duokgu call”. The electronic device 100 mayperform a multi-microphone process associated with second audio datacorresponding to “Oh Duokgu call” collected using the plurality ofmicrophones Mic1 to MicN. The electronic device 100 may control toperform voice recognition of third audio data, experiencing themulti-microphone process, to perform a call connection function.

According to various embodiments of the present disclosure, the firstvoice recognition model 51 may include a plurality of utterancecharacter information and a plurality of classification information.Accordingly, a wake-up command of the voice recognition function may bedefined by at least one. Furthermore, an authentication function of thewake-up command of the voice recognition function may be defined byclassification information of a plurality of speakers. The audio dataprocessing model 180 may provide a screen associated with an input or achange or adjustment of the wake-up command. The audio data processingmodel 180 may register a wake-up command, which is input on a wake-upcommand input screen, as specific audio data at the first voicerecognition model 51. According to various embodiments of the presentdisclosure, the wake-up command may be defined only by speakerclassification information without specified utterance characterinformation. If specific audio data is collected, the audio dataprocessing model 180 may determine whether the specific audio datacorresponds to speaker classification information of an authenticatedperson and may control activation of the voice recognition functionaccording to the determination result.

The second voice recognition model 53 may be a model that supports voicerecognition of a variety of audio data of a speaker. For example, thesecond voice recognition model 53 may be a model that recognizes a voicein the form of letters or words, vocabularies, and morphemes pronouncedin Korean. According to various embodiments of the present disclosure,the second voice recognition model 53 may be a model which recognizes avoice in the form of letters or words, vocabularies, and morphemespronounced in at least one of English, Japanese, Spanish, French,German, Hindustani, and the like. If comparison of the specific audiodata is completed through the first voice recognition model 51, thesecond voice recognition model 53 may be provided to a device componentthat performs the voice recognition function. The second voicerecognition model 53 may be implemented to be different from the firstvoice recognition model 51 or may include the first voice recognitionmodel 51.

According to various embodiments of the present disclosure, the firstvoice recognition model 51 or the second voice recognition model 53 maybe stored (or disposed) at different storage areas. For example, thefirst voice recognition model 51 may be disposed at an audio codec (or astorage space which the audio codec can access), and the second voicerecognition model 53 may be disposed at the audio data processing module180 (or a storage space which the audio data processing module 180 canaccess). According to various embodiments of the present disclosure, thefirst voice recognition model 51 may be disposed at the low-powerprocessing module 170 (or a storage space which the low-power processingmodule 170 can directly access).

According to various embodiments of the present disclosure, the memory150 may include a buffer for temporarily storing audio data with regardto processing audio data that the microphone module MIC collects. Thebuffer may store audio data which a default microphone collects or audiodata which the plurality of microphones Mic1 to MicN collects. In thisregard, at least one of the size of buffer or the number of buffers maybe adjusted according to a control of the audio data processing module180. The above-described buffer may be included in the memory 150.Alternatively, the buffer may be implemented to be independent of thememory 150.

The low-power processing module 170 may collect a signal associated withat least one sensor that the electronic device 100 includes. Forexample, the low-power processing module 170 may activate at least onemicrophone of the microphone module MIC and may collect audio data.Power consumption of the low-power processing module 170 may be lessthan those of the audio codec 130 and the audio data processing module180, and the low-power processing module 170 may be designed to operatethe microphone module MIC. For example, the low-power processing module170 may include circuit modules associated with the voice recognitionfunction and a signal line(s). According to an embodiment of the presentdisclosure, the low-power processing module 170 may be designed toperform at least one control process of activation of at least onemicrophone, collection of audio data, comparison between collected audiodata and specific audio data, and a multi-microphone control processaccording to the comparison result.

The microphone module MIC may include the plurality of microphones Mic1to MicN. For example, the microphone module MIC may include a firstmicrophone Mic1 and a second microphone Mic2. Either the firstmicrophone Mic1 or the second microphone Mic2 may be activated toperform the power saving function in executing the voice recognitionfunction. Alternatively, the first microphone Mic1 and the secondmicrophone Mic2 may be activated to perform the detail recognitionfunction in executing the voice recognition function. At least one pieceof audio data that the first microphone Mic1 and the second microphoneMic2 collect may be supplied to at least one of the audio codec 130, thelow-power processing module 170, or the audio data processing module180. Audio data that at least one of the microphones Mic1 to MicNcollects may be temporarily stored at the buffer of the memory 150.

The processor 160 may receive instructions from the above-describedother components (e.g., the communication interface 110, theinput/output interface 120, the display 140, the memory 150, the audiodata processing module 180, and the like) through the bus 190, maydecode the received instructions, and may perform data processing oroperations according to the decoded instructions.

The audio data processing module 180 may process and transfer dataassociated with an operation of the electronic device 100 and mayprocess and transfer a control signal. According to an embodiment of thepresent disclosure, the audio data processing module 180 may support atleast one of an activation control of the microphone module MICassociated with execution of the voice recognition function, a wake-upcommand process, a multi-microphone control process, a voice recognitionfunction process, and an additional function execution process accordingto voice recognition. According to an embodiment of the presentdisclosure, the audio data processing module 180 may include a firstsignal processing module, a second signal processing module, amulti-channel signal processing module, a DOA decision unit, or abeamforming/noise cancelling module. The first signal processing modulemay include at least one of a single channel signal processing module ora first voice recognition module. The second signal processing modulemay include a multi-channel signal processing module or a second voicerecognition module. Each module of the above-described audio dataprocessing module 180 may be implemented using at least one processor160. At least a portion of the audio data processing module 180 with theabove-described configuration may be disposed in at least one of theaudio codec 130 or the low-power processing module 170.

According to various embodiments of the present disclosure, if audiodata, corresponding to execution of a specific function, from amongcollected audio data, is collected, the audio data processing module 180may control to perform a relevant function. According to an embodimentof the present disclosure, the audio data processing module 180 mayexecute voice recognition of the collected audio data. The audio dataprocessing module 180 may control to perform a specific functioncorresponding to voice-recognized audio data.

According to various embodiments of the present disclosure, thecommunication interface 110 may receive broadcasting data, based on abroadcasting receiving unit. When outputting the received broadcastingdata, the audio data processing module 180 may support the voicerecognition function with respect to audio data included in thebroadcasting data. The audio data processing module 180 may controlactivation of the voice recognition function and execution of a functionaccording to voice recognition, if there is collected a specificactivation command (e.g., a command associated with activation of thevoice recognition function and a command (wakeup command) for waking upthe voice recognition function) and audio data corresponding to voicerecognition are collected. For example, the audio data processing module180 may control a channel change of the broadcasting receiving unit,based on voice recognition. The activation command may correspond tospecific audio data, for example, specific audio data set to theelectronic device 100 or specific voice data set by a user.

According to various embodiments of the present disclosure, specificaudio data corresponding to the activation command may be, for example,“Hi Samsung”. As audio data collected next to the specific audio data,the function execution command may be, for example, “Channel Change 11”,“Channel 5”, and the like. Based on the audio data collected next to thespecific audio data, the communication interface 110 may change achannel to channel 11 or may change the channel to channel 5.

According to various embodiments of the present disclosure, the audiodata processing module 180 may receive a result of the multi-microphonecontrol process from the audio codec 130. The audio data processingmodule 180 may perform the voice recognition function, based on thesecond voice recognition model 53.

According to various embodiments of the present disclosure, the audiodata processing module 180 may receive the wakeup command from thelow-power processing module 170. The audio data processing module 180may perform the multi-microphone control process and the voicerecognition function that is based on the second voice recognition model53.

According to various embodiments of the present disclosure, the audiodata processing module 180 may receive the wakeup command, which thelow-power processing module 170 transfers, as well as a result of themulti-microphone control process which the audio codec 130 transfers.The audio data processing module 180 may perform the voice recognitionfunction that is based on the second voice recognition model 53.

According to various embodiments of the present disclosure, the audiodata processing module 180 may receive a value (hereinafter referred toas “DOA decision value”) for deciding a direction of arrival, from thelow-power processing module 170. The audio data processing module 180may perform the multi-microphone control process in response to the DOAdecision value and may process the voice recognition function that isbased on the second voice recognition model 53.

According to various embodiments of the present disclosure, the audiodata processing module 180 may process a wakeup command search function,which is based on the first voice recognition model 51 of the low-powerprocessing module 170, and a voice recognition function which is basedon the audio codec 130.

According to various embodiments of the present disclosure, theelectronic device 100 may include a first processor which collectsspecific audio data with regard to a voice recognition function andgenerates a wakeup command, a multi-microphone processing module whichperforms a multi-microphone process associated with the collected audiodata in response to the wakeup command, and a second processor whichexecutes voice recognition with respect to audio data experiencing themulti-microphone process. The first processor, the multi-microphoneprocessing module, and the second processor may be disposed in one ofthe audio codec 130, the low-power processing module 170, and the audiodata processing module 180.

According to various embodiments of the present disclosure, theelectronic device 100 may include a first processor which collectsspecific audio data with regard to a voice recognition function andgenerates a wakeup command, a DOA decision unit, which determinesdirections of arrival associated with a plurality of microphones Mic1 toMicN in response to the wakeup command, a beamforming/noise cancelingmodule, which applies beamforming or noise canceling according to thedirections of arrival determined thus, and a second processor whichexecutes voice recognition with respect to beam-formed or noise-canceledaudio data. The first processor, the DOA decision unit, thebeamforming/noise canceling module, and the second processor may bedisposed in one of the audio codec 130, the low-power processing module170, and the audio data processing module 180. A module according tovarious embodiments of the present disclosure may be hardware, firmware,software, or a combination of at least two thereof.

Hereinafter, arrangement of the above-described processors and devicecomponents is more fully described below with reference to accompanyingdrawings.

FIG. 2 illustrates an electronic device which uses microphones, based onan audio codec and an audio data processing module, according to anembodiment of the present disclosure.

Referring to FIG. 2, an electronic device 100 associated with using of aplurality of microphones according to an embodiment of the presentdisclosure may contain the audio codec 130 which includes a first signalprocessing module 10 (including a single channel signal processingmodule 11 and a first voice recognition module 12) and a multi-channelsignal processing module 30, the audio data processing module 180 whichincludes a second signal processing module 20 (including apre-processing module 21 and a second voice recognition module 22), anda plurality of microphones Mic1 to MicN.

The audio codec 130 may include the first signal processing module 10and the multi-channel signal processing module 30. The first signalprocessing module 10 of the first audio codec 130 may control toactivate a first microphone Mic1 corresponding to a default microphoneaccording to setting. For example, if a power saving function is set ata state in which activation of a voice recognition function isrequested, the first signal processing module 10 may control to activatethe first microphone Mic1. The first signal processing module 10 mayoperate a first voice recognition model 51 stored in the memory 150. Thefirst signal processing module 10 may perform voice recognition of firstaudio data that the first microphone Mic1 collects. The first signalprocessing module 10 may determine whether the collected first audiodata is specific audio data corresponding to the first voice recognitionmodel 51. When the collected first audio data is the specific audiodata, the first signal processing module 10 may transfer a wakeupcommand, set to activate the multi-channel signal processing module 30,to the multi-channel signal processing module 30.

The first signal processing module 10 may include the single channelsignal processing module 11 and the first voice recognition module 12,and additionally or generally, may include the first voice recognitionmodel 51. The single channel signal processing module 11 may correct thefirst audio data which the first microphone Mic1 collects. For example,the single channel signal processing module 11 may perform at least aportion of functions capable of processing an audio signal, such asadaptive echo canceler (AEC), noise suppression (NS), end-pointdetection (EPD), automatic gain control (AGC), and the like. As regardssupporting a low-power operation, the first signal processing module 10may omit the whole pre-processing function, or may control to perform aportion of the pre-processing function. The first signal processingmodule 10 may be driven using power different from that of the secondsignal processing module 20, for example, power less than thatassociated with an operation of the second signal processing module 20.In the case where designed such that a portion of the preprocessingfunction is applied, the single channel signal processing module 11 maypreprocess audio data collected according to a relevant design. Thesingle channel signal processing module 11 may transfer the preprocessedaudio data to the first voice recognition module 12. In the case wherethe preprocessing function is omitted with regard to low-power driving,a configuration, associated with the preprocessing function, of thesingle channel signal processing module 11 may be omitted. In this case,the collected audio data may be directly processed by the first voicerecognition module 12.

The first voice recognition module 12 may analyze whether audio datacollected through loading and operating of the first voice recognitionmodel 51 is specific audio data (or whether similarity between thecollected audio data and a trained statistical model is within aspecific error range). The first voice recognition model 51 may bestored at a memory 150 and may be referred by the first voicerecognition model 12 or may be mounted at the first signal processingmodule 10. The first voice recognition model 12 may generate a wakeupcommand (or activation command) when the specific audio data iscollected. The first voice recognition model 12 may transfer the wakeupcommand to the multi-channel signal processing module 30.

If receiving the wakeup command from the first signal processing module10, the multi-channel signal processing module 30 included in the audiocodec 130 may control to activate a plurality of microphones Mic1 toMicN included in a microphone module MIC. The multi-channel signalprocessing module 30 may apply a multi-microphone processing function tosecond audio data collected by the microphones Mic1 to MicN to generatethird audio data experiencing multi-microphone processing. For example,the multi-channel signal processing module 30 may include a DOAdetection unit, a beamforming unit, a noise reduction unit, an errorcancellation unit, and the like. The multi-channel signal processingmodule 30 may provide directivity of a voice obtaining direction ofsecond audio data collected to generate third audio data of which theSINR (signal to interference noise ratio) is enhanced. The DOA detectionunit may detect a direction of arrival associated with the second audiodata collected. A function of detecting the direction of arrival may bea function of detecting a direction of a selected voice. The beamformingunit may perform beamforming in which a sound of a specific direction isobtained by applying a filter corresponding to a parameter, which iscalculated using a voice direction value detected according to thedirection-of-arrival function, to second audio data received from theplurality of microphones Mic1 to MicN. The noise reduction unit mayperform noise suppression (NS) by suppressing obtaining of a sound of aspecific direction. The echo cancellation unit may perform echocancelling of the second audio data collected. The multi-channel signalprocessing module 30 may apply at least one of the above-describedmulti-microphone processing functions to the second audio data togenerate the third audio data.

The third audio data processed by the multi-channel signal processingmodule 30 may be provided to the second signal processing module 20 ofthe audio data processing module 180. At this time, the second audiodata which the multi-channel signal processing module 30 collects mayhave a more accurate voice signal characteristic according to themulti-microphone processing function.

The second signal processing module 20 of the audio data processingmodule 180 may receive the third audio data, to which themulti-microphone processing function is applied, from the multi-channelsignal processing module 30. The second signal processing module 20 mayperform a voice recognition function of the third audio data. The secondsignal processing module 20 may use a second voice recognition model 53stored at the memory 150.

The second signal processing module 20 may perform a specific functionin response to a voice recognition result of the third audio data. Forexample, the second signal processing module 20 may control to perform asearch function in which the voice recognition result is used as asearch word. According to an embodiment of the present disclosure, thesecond signal processing module 20 may search for and output data,associated with a search word corresponding to the voice recognitionresult, from the memory 150. According to an embodiment of the presentdisclosure, the second signal processing module 20 may transmit thevoice recognition result to a specific server device and may receive andoutput information corresponding to the voice recognition result fromthe specific server device. According to various embodiments of thepresent disclosure, the second signal processing module 20 may controlto activate a specific function corresponding to the voice recognitionresult.

The second signal processing module 20 may include the preprocessingmodule 21 and the second voice recognition module 22, and additionallyor generally, may include the second voice recognition model 53. Thepreprocessing module 21 may employ at least one of various functionscapable of processing audio signals, such as adaptive echo canceler(AEC), noise suppression (NS), end-point detection (EPD), automatic gaincontrol (AGC), and the like. For example, if an output signal isgenerated while an application (a call application, a ring toneapplication, a music player application, a camera application, and thelike) capable of generating an output signal during a voice input isrunning, the preprocessing module 21 may apply the AEC function for echoprocessing to the output signal. Audio data preprocessed by thepreprocessing module 21, for example, audio data obtained bypreprocessing the third audio data from the multi-channel signalprocessing module 30 may be transferred to the second voice recognitionmodule 22.

The second voice recognition module 22 may transfer a voice recognitionresult of audio data voice-recognized (or additionally preprocessed)based on the second voice recognition model 53, to the audio dataprocessing module 180. Alternatively, the second voice recognitionmodule 22 may transfer the voice recognition result to a devicecomponent where the second signal processing module 20 is disposed. Thedevice component which receives the voice recognition result may controlto execute a specific function in which the voice recognition result isused as a function execution command.

According to various embodiments of the present disclosure, the secondsignal processing module 20 may perform the voice recognition functionusing the server device 106. For example, when receiving the third audiodata, the second voice recognition module 22 of the second signalprocessing module 20 may control to activate a communication interface110 to form a communication channel with a server device supporting thevoice recognition function. The second signal processing module 20 maytransfer the collected third audio data to the server device 106 and mayreceive a voice recognition result from the server device 106. Thesecond signal processing module 20 may perform a preprocessing operationof the third audio data. In this case, the third audio data transmittedto a server device may be third audio data preprocessed.

A first microphone Mic1 of the plurality of microphones Mic1 to MicN maybe activated according to a control of the first signal processingmodule 10. The first microphone Mic1 may provide the first signalprocessing module 10 with first audio data collected. In the case wherethe first audio data collected by the first microphone Mic1 is specificaudio data, the microphones Mic2 to MicN may be activated according to acontrol of the multi-channel signal processing module 30. Alternatively,the plurality of microphones Mic1 to MicN may be activated according toa control of the first signal processing module 10. The first audio datacollected by the first microphone Mic1 may be also provided to themulti-channel signal processing module 30. Accordingly, the firstmicrophone Mic1 may include a signal line for supplying audio data tothe first signal processing module 10 and a signal line for supplyingaudio data to the multi-channel signal processing module 30. The firstmicrophone Mic1 may change a provider of audio data according to acontrol of the first signal processing module 10 or the multi-channelsignal processing module 30. Second to N-th microphones Mic2 to MicN ofthe microphone module MIC may be configured to supply the second audiodata collected to the multi-channel signal processing module 30.Accordingly, the first microphone Mic1 used as a default microphone maybe controlled by the first signal processing module 10, and the secondto N-th microphones Mic2 to MicN may be controlled by the multi-channelsignal processing module 30.

As described above, the electronic device 100 according to variousembodiments of the present disclosure may detect a wakeup command usingthe first microphone Mic1, and when the wakeup command is detected, theelectronic device 100 may activate the microphones Mic2 to MicN tocollect audio data to which the multi-microphone processing function isapplied. Accordingly, the electronic device 100 according to variousembodiments of the present disclosure may collect and apply accurateaudio data in an actual voice recognition section while saving powerusing a default microphone.

FIG. 3 illustrates an electronic device which uses microphones based ona low-power processing module and an audio data processing module,according to various embodiments of the present disclosure.

Referring to FIG. 3, an electronic device 100 associated with anoperation of microphones, according to an embodiment of the presentdisclosure may contain a low-power processing module 170 including afirst signal processing module 10, the audio data processing module 180including the multi-channel signal processing module 30 and the secondsignal processing module 20, and a plurality of microphones Mic1 toMicN.

The low-power processing module 170 may control activation of a firstmicrophone Mic1 and collection of first audio data, based on the firstsignal processing module 10. The low-power processing module 170 may usethe first microphone Mic1 if execution of a voice recognition functionis requested or the first microphone Mic1 is set as a defaultmicrophone, and may perform activation of the first microphone Mic1 andcollection of the first audio data. According to an embodiment of thepresent disclosure, with regard to voice recognition of first audio datacollected through the first microphone Mic1, the low-power processingmodule 170 may use the first microphone Mic1 according to a control ofthe audio data processing module 180.

As regards voice recognition of the first audio data collected throughthe first microphone Mic1 activated, the first signal processing module10 of the low-power processing module 170 may load or activate a firstvoice recognition model 51 stored in the memory 150. The first signalprocessing module 10, as described with reference to FIG. 1, maydetermine whether the first audio data is the same or similar tospecific audio data. The first signal processing module 10 may determinewhether the first audio data is the specific audio data corresponding tothe first voice recognition model 51.

The first signal processing module 10 may transfer a wakeup command tothe audio data processing module 180 according to a result of analyzingthe first audio data. For example, the first signal processing module 10may transfer the wakeup command to the multi-channel signal processingmodule 30 of the audio data processing module 180. As regards a voicerecognition function, the first signal processing module 10 of thelow-power processing module 170 may have a sleep state or a low-poweroperating state before transferring the wakeup command. For example,when the first signal processing module 10 of the low-power processingmodule 170 detects specific audio data, a display 140 may remain at aturn-off state in response to a control of the audio data processingmodule 180. If first audio data corresponding to specific audio data (ora specific error range between first audio data and a trainedstatistical model) is collected, the first signal processing module 10may change a transmission path of audio data that the first microphoneMic1 collects. For example, the first signal processing module 10 maycontrol to transfer audio data, which the first microphone Mic1collects, to the multi-channel signal processing module 30 of the audiodata processing module 180. Furthermore, the first signal processingmodule 10 may transfer authority for activating or inactivating thefirst microphone Mic1 to the audio data processing module 180.

If receiving the wakeup command from the low-power processing module170, the audio data processing module 180 may transition from the sleepstate to a wake state. The audio data processing module 180 may activatethe multi-channel signal processing module 30 and the second signalprocessing module 20 with regard to supporting the voice recognitionfunction.

The multi-channel signal processing module 30 of the audio dataprocessing module 180 may control an operation of a microphone moduleMIC in response to the wakeup command from the low-power processingmodule 170. For example, second to N-th microphones Mic2 to MicN may beset to an active state according to a control of the multi-channelsignal processing module 30, respectively. The multi-channel signalprocessing module 30 may determine a transmission path of audio dataassociated with the first microphone Mid and may receive audio data thatthe first microphone Mic1 collects. When receiving the wakeup command,the multi-channel signal processing module 30 may obtain authority foruse (e.g., authority for activating or inactivating the first microphoneMic1) of the first microphone Mic1. The multi-channel signal processingmodule 30 may generate third audio data by applying a multi-microphoneprocessing function to pieces of second audio data that microphones Mic1to MicN included in the microphone module MIC collect. The multi-channelsignal processing module 30 may transfer the third audio data thusgenerated to the second signal processing module 20.

In the case where a state of the audio data processing module 180 ischanged according to an input of the wakeup command, the second signalprocessing module 20 may use a second voice recognition model 53 storedat the memory 150 (or mounted at a signal processing module). The secondsignal processing module 20 may perform voice recognition of the thirdaudio data, which the multi-channel signal processing module 30transfers, based on the second voice recognition model 53. The secondsignal processing module 20 may control to perform a specific functionbased on a voice-recognized result value. For example, as describedabove, the second signal processing module 20 may perform a searchfunction according to a voice recognition result. The second signalprocessing module 20 may control to end an application associated withthe voice recognition function upon receiving an end event of theapplication. The second signal processing module 20 may control totransfer the application end event to the multi-channel signalprocessing module 30 to inactivate a portion of the plurality ofmicrophones Mic1 to MicN.

The first microphone Mic1 of the microphone module MIC may be coupled tothe first signal processing module 10 of the low-power processing module170. Furthermore, the first microphone Mic1 may be coupled to themulti-channel signal processing module 30 of the audio data processingmodule 180. When activated in response to a control of the first signalprocessing module 10 of the low-power processing module 170, the firstmicrophone Mic1 may collect first audio data and may transfer the firstaudio data thus collected to the first signal processing module 10.Under a control of the multi-channel signal processing module 30, thefirst microphone Mic1 may collect second audio data together with othermicrophones and may provide the second audio data thus collected to themulti-channel signal processing module 30. Second to N-th microphonesMic2 to MicN of the microphone module MIC may be coupled to themulti-channel signal processing module 30. In response to a control ofthe multi-channel signal processing module 30, the second to N-thmicrophones Mic2 to MicN may collect pieces of second audio data and maytransfer the pieces of the second audio data to the multi-channel signalprocessing module 30. The second to N-th microphones Mic2 to MicN may beinactivated in response to a control of the multi-channel signalprocessing module 30. The first microphone Mic1 may be also deactivatedin response to a control of the multi-channel signal processing module30. The first signal processing module 10 of the low-power processingmodule 170 may obtain authority for controlling the first microphoneMic1 set to an inactive state.

According to various embodiments of the present disclosure, the audiodata processing module 180 may be implemented with an audio codec 130.Accordingly, the audio codec 130 may include the multi-channel signalprocessing module 30 and the second signal processing module 20. Whenreceiving a wakeup command from the first signal processing module 10 ofthe low-power processing module 170, the audio codec 130 may use themicrophone module MIC to collect pieces of second audio data. Themulti-channel signal processing module 30 of the audio codec 130 mayapply a multi-microphone processing function to the second audio datathus collected to generate third audio data and may transfer the thirdaudio data to the second signal processing module 20. The second signalprocessing module 20 of the audio codec 130 may execute voicerecognition of the third audio data and may transfer a voice recognitionresult to the audio data processing module 180. The audio dataprocessing module 180 may perform a predetermined function, based on thevoice recognition result which the audio codec 130 transfers.

FIG. 4 illustrates an electronic device which uses microphones based ona low-power processing module, an audio data processing module, and anaudio codec according to various embodiments of the present disclosure.

Referring to FIG. 4, an electronic device 100 relating to microphonemanagement according to an embodiment of the present disclosure mayinclude the audio codec 130 including the multi-channel signalprocessing module 30, the low-power processing module 170 including thefirst signal processing module 10, and the audio data processing module180 including the second signal processing module 20, and a plurality ofmicrophones Mic1 to MicN.

As regards execution of a voice recognition function, the low-powerprocessing module 170 may use the first signal processing module 10. Thefirst signal processing module 10 of the low-power processing module 170may control to active a first microphone Mic1 if execution of a powersaving function of a voice recognition function is set or requested. Thefirst signal processing module 10 of the low-power processing module 170may compare audio data, which the first microphone Mic1 collects, with afirst voice recognition model 51 to detect specific audio data (orsimilarity between the audio data and a trained statistical model). Whenthe specific audio data is detected, the first signal processing module10 may transfer a wakeup command to the multi-channel signal processingmodule 30 of the audio codec 130. Furthermore, under a control of thefirst signal processing module 10 of the low-power processing module170, a transmission path of audio data collected by the first microphoneMic1 may be changed so as to be transferred to the multi-channel signalprocessing module 30 of the audio codec 130. Under a control of thefirst signal processing module 10 of the low-power processing module170, a wakeup command may be transferred to the audio data processingmodule 180 so as to activate the second signal processing module 20.

The audio codec 130 may activate the multi-channel signal processingmodule 30 in response to an input of the wakeup command from the firstsignal processing module 10 of the low-power processing module 170. Themulti-channel signal processing module 30 of the audio codec 130 mayactivate second to N-th microphones Mic2 to MicN. The multi-channelsignal processing module 30 of the audio codec 130 may collect secondaudio data using a microphone module MIC. The multi-channel signalprocessing module 30 of the audio codec 130 may generate third audiodata by applying a multi-microphone processing function to the secondaudio data collected. The multi-channel signal processing module 30 ofthe audio codec 130 may transfer the third audio data to the secondsignal processing module 20 of the audio data processing module 180.

The second signal processing module 20 of the audio data processingmodule 180 may execute voice recognition of the third audio data whichthe audio codec 130 transfers. The second signal processing module 20may preprocess the third audio data. The second signal processing module20 may perform a voice recognition function of the preprocessed thirdaudio data, based on a second voice recognition model 53. The secondsignal processing module 20 may control to execute a specific functionin response to a voice recognition result.

The first microphone Mic1 of the microphone module MIC may be coupled tothe first signal processing module 10 of the low-power processing module170. Furthermore, the first microphone Mic1 may be coupled to themulti-channel signal processing module 30 of the audio codec 130. Thefirst microphone Mic1 may be activated according to a control of thefirst signal processing module 10, and a transmission path may bechanged after first audio data corresponding to specific audio data iscollected. The first microphone Mic1 may provide the multi-channelsignal processing module 30 of the audio codec 130 with audio data thatis collected after the first audio data corresponding to the specificaudio data is collected. Second to N-th microphones Mic2 to MicN may becoupled to the multi-channel signal processing module 30 of the audiocodec 130 and may collect pieces of second data in response to a controlof the multi-channel signal processing module 30.

As described above, the electronic device 100 according to variousembodiments of the present disclosure may include the first microphoneMic1 which collects first audio data, the first signal processing module10 which determines whether the first audio data includes specific audiodata, the multi-channel signal processing module 30 which, when thespecific audio data is detected, collects second audio data using aplurality of microphones and performs multi-microphone processingassociated with the second audio data, and the second signal processingmodule 20 which executes voice recognition of the third audio dataexperiencing the multi-microphone processing.

According to various embodiments of the present disclosure, the firstsignal processing module 10 may include the first preprocessing unitwhich performs at least a portion of a plurality of preprocessingfunctions of the first audio data, the first voice recognition module 12which executes voice recognition of the first audio data, and the firstvoice recognition model 51 which supports regulation voltage of thefirst audio data.

According to various embodiments of the present disclosure, the firstvoice recognition model 51 may include at least one of utterancecharacter information and speaker classification informationcorresponding to the specific audio data.

According to various embodiments of the present disclosure, when thespecific audio data is detected, the first signal processing module 10may generate a wakeup command and may transfer the wakeup command to themulti-channel signal processing module 30.

According to various embodiments of the present disclosure, themulti-channel signal processing module 30 may activate the plurality ofmicrophones Mic1 to MicN in response to an input of the wakeup command.

According to various embodiments of the present disclosure, the secondsignal processing module 20 may include the second preprocessing unitwhich performs a plurality of preprocessing functions of the third audiodata, the second voice recognition module 22 which executes voicerecognition of the third audio data, and the second voice recognitionmodel 53 which supports voice recognition of the third audio data.

According to various embodiments of the present disclosure, the secondsignal processing module 20 may control execution of a specific functioncorresponding to the voice-recognized result.

According to various embodiments of the present disclosure, themulti-channel signal processing module 30 may include at least one of aDOA detection unit configured to detect directions of arrival associatedwith the second audio data, a beamforming processing unit configured toperform beamforming according to a detection of the directions ofarrival, a noise suppression unit configured to suppress a noise bysuppressing obtaining of a sound of a specific direction with respect tothe pieces of the second audio data, and an echo cancellation unitconfigured to perform echo cancellation of the pieces of the secondaudio data.

According to various embodiments of the present disclosure, theelectronic device 100 may include the audio codec 130 in which the firstsignal processing module 10 and the multi-channel signal processingmodule 30 are disposed and the audio data processing module 180 in whichthe second signal processing module 20 is disposed.

According to various embodiments of the present disclosure, theelectronic device 100 may include the low-power processing module 170,including the first signal processing module 10, and the audio dataprocessing module 180, including the multi-channel signal processingmodule 30 and the second signal processing module 20.

According to various embodiments of the present disclosure, theelectronic device 100 may include the low-power processing module 170,including the first signal processing module 10, the audio codec 130,including the multi-channel signal processing module 30, and the audiodata processing module 180, including the second signal processingmodule 20.

FIG. 5 illustrates an electronic device which supports microphoneintegrated employment based on a low-power processing module and anaudio codec, according to an embodiment of the present disclosure.

Referring to FIG. 5, the electronic device relating to microphoneintegrated employment may include the low-power processing module 170,which includes the first signal processing module 10 and the DOAdecision unit 40, and the audio codec 130, which includes thebeamforming/noise suppression module 50 and the second signal processingmodule 20.

As regards executing a voice recognition function, the low-powerprocessing module 170 may employ the first signal processing module 10and the DOA decision unit 40. The first signal processing module 10 ofthe low-power processing module 170 may control to activate a microphonemodule MIC when execution of a detail recognition function of a voicerecognition function is set or requested. Alternatively, as regardsexecuting the detail recognition function, by default, the first signalprocessing module 10 of the low-power processing module 170 may controlto activate the microphone module MIC. The first signal processingmodule 10 of the low-power processing module 170 may compare first audiodata, which a first microphone Mic1 collects, from among pieces of firstaudio data collected by the microphone module MIC with a first voicerecognition model 51 to detect specific audio data. When the specificaudio data is detected, the first signal processing module 10 of thelow-power processing module 170 may transfer a wakeup command to the DOAdecision unit 40.

The DOA decision unit 40 of the low-power processing module 170 maycollect pieces of first audio data from the microphone module MIC thatthe first signal processing module 10 activates. In this operation, theDOA decision unit 40 of the low-power processing module 170 maytemporarily store (buffer) the collected pieces of the first audio data.If receiving a wakeup command from the first signal processing module 10of the low-power processing module 170, the DOA decision unit 40 of thelow-power processing module 170 may determine a direction of arrival orgenerate information associated with a microphone array (MA), based onthe first audio data. According to an embodiment of the presentdisclosure, the DOA decision unit 40 of the low-power processing module170 may determine a sound obtaining direction based on the first audiodata. The DOA decision unit 40 of the low-power processing module 170may calculate a parameter associated with weighting a plurality ofmicrophones Mid to MicN included in the microphone module MIC.

According to an embodiment of the present disclosure, the DOA decisionunit 40 of the low-power processing module 170 may define weightparameters of a first microphone Mic1 and a second microphone Mic2differently, based on a result of analyzing audio data collected by themicrophone module MIC. The DOA decision unit 40 of the low-powerprocessing module 170 may transfer the calculated weight parameters tothe beamforming/noise suppression module 50 of the audio codec 130performing beam-forming/noise-suppression. The low-power processingmodule 170 may use audio data, which the first microphone Mic1 collectsin real time, to determine specific audio data and to determine adirection of arrival based on buffering. According to variousembodiments of the present disclosure, the low-power processing module170 may use the first microphone Mic1 as being dedicated to determinethe specific audio data. The low-power processing module 170 may usesecond to N-th microphones Mic2 to MicN to determine a direction ofarrival. As regards the above condition, the low-power processing module170 may set a plurality of microphones to a waiting state to determine adirection of arrival.

The beamforming/noise suppression module 50 of the audio codec 130 mayprocess a weight parameter (a parameter associated with a beamformingdirection or noise suppression) received from the DOA decision unit 40of the low-power processing module 170. For example, thebeamforming/noise suppression module 50 may apply different weights to aplurality of microphones Mic1 to MicN, based on a weight value of theweight parameter. The beamforming/noise suppression module 50 may applydifferent weights to audio data collected by the microphones Mic1 toMicN (i.e., different weights are applied to the microphones Mic1 toMicN, respectively), and may transfer the weighed audio data to thesecond signal processing module 20 of the audio codec 130. According toan embodiment of the present disclosure, the beamforming/noisesuppression module 50 may set a weight of the first microphone Mic1 to,for example, 0.3, a weight of the second microphone Mic2 to, forexample, 0.5, and a weight of the N-th microphone MicN to, for example,0.2, based on the weight parameter. Weights thus set may be changed inreal time or periodically according to the weight parameter that the DOAdecision unit 40 provides.

The second signal processing module 20 of the audio codec 130 mayreceive beam-formed or noise-suppressed audio data from thebeamforming/noise suppression module 50 of the audio codec 130 and mayperform voice recognition of the beam-formed or noise-suppressed audiodata. In this operation, the second signal processing module 20 of theaudio codec 130 may preprocess the collected audio data and may performvoice recognition based on a second voice recognition model 53. Thesecond signal processing module 20 of the audio codec 130 may transfer avoice recognition result to the audio data processing module 180.Alternatively, the audio codec 130 may control to perform a specificfunction according to a voice recognition result that the second signalprocessing module 20 outputs.

The first microphone Mic1 of the microphone module MIC may be coupled tothe first signal processing module 10 of the low-power processing module170, the DOA decision unit 40 of the low-power processing module 170,and the beamforming/noise suppression module 50 of the audio codec 130.The second to N-th microphones Mic2 to MicN may be coupled to the DOAdecision unit 40 of the low-power processing module 170 and thebeamforming/noise suppression module 50 of the audio codec 130. Themicrophone module MIC may be activated according to a control of thefirst signal processing module 10 of the audio codec 130, and atransmission path may be changed according to collection of first audiodata corresponding to specific audio data. For example, the firstmicrophone Mic1 may transfer the first audio data corresponding tospecific audio data to the firs signal processing module 10 of thelow-power processing module 170 and the DOA decision unit 40 of thelow-power processing module 170. The second to N-th microphones Mic2 toMicN may transfer the first audio data to the DOA decision unit 40 ofthe low-power processing module 170. Afterwards, the second audio datamay be transferred to the beamforming/noise suppression module 50 of theaudio codec 130, and the beamforming/noise suppression module 50 of theaudio codec 130 may apply at least one of beamforming and noisesuppression thereto so as to be converted into parameter-processed audiodata. The parameter-processed audio data may be transferred to thesecond signal processing module 20 of the audio codec 130.

As performing a voice recognition function according to theabove-described manner, the electronic device 100 may perform voicerecognition and functions seamlessly. For example, audio data such as“Hi Samsung, Broadcasting Channel 5” may be collected by a plurality ofmicrophones Mic1 to MicN. “Hi Samsung” collected by the first microphoneMic1 may be transferred to the first signal processing module 10 todetermine whether it is specific audio data. In this operation, thesecond to N-th microphones Mic2 to MicN may buffer and store first audiodata corresponding to “Hi Samsung”. When receiving a wakeup command fromthe first signal processing module 10, the DOA decision unit 40 of thelow-power processing module 170 may determine a weight parameter, basedon the buffered “Hi Samsung”. In the case where “Hi Samsung” is not thespecific audio data, the DOA decision unit 40 of the low-powerprocessing module 170 may replace it with another audio data receivedlater. Another embodiment may be also possible. A weight parameter thatthe DOA decision unit 40 calculates may be transferred to thebeamforming/noise suppression module 50.

The beamforming/noise suppression module 50 of the audio codec 130 mayweight, for example, “Broadcasting Channel 5” and may transfer theweighted audio data to the second signal processing module 20. Thesecond signal processing module 20 may perform preprocessing and voicerecognition of the weighted audio data and may calculate a result ofperforming the preprocessing and voice recognition. Under a control ofthe audio data processing module 180 receiving a voice recognitionresult from the audio codec 130, a broadcasting receiving unit may beactivated when “Broadcasting Channel 5” is recognized as a voicerecognition result, thereby allowing a broadcasting channel to be tunedto broadcasting channel 5. According to various embodiments of thepresent disclosure, the beamforming/noise suppression module 50 mayreceive information of a section corresponding to the specific audiodata recognized as “Hi Samsung”, from the DOA decision unit 40. Thebeamforming/noise suppression module 50 may weight audio data collectedafterwards, without processing the specific audio data. Accordingly, thebeamforming/noise suppression module 50 may transfer, to the secondsignal processing module 20, audio data about “Broadcasting Channel 5”except “Hi Samsung”.

FIG. 6 illustrates an electronic device which supports low-powermicrophone integrated use based on a low-power processing module and anaudio codec, according to an embodiment of the present disclosure.

Referring to FIG. 6, an electronic device associated with microphoneintegrated use, according to an embodiment of the present disclosure mayinclude the low-power processing module 170, which includes the firstsignal processing module 10, the audio codec 130, which includes the DOAdecision unit 40, the beamforming/noise suppression module 50, and thesecond signal processing module 20.

When execution of a detail recognition function of a voice recognitionfunction is set or requested, the first signal processing module 10 ofthe low-power processing module 170 may control to activate themicrophone module MIC. Alternatively, the first signal processing module10 of the low-power processing module 170 may detect specific audio databy comparing audio data, which a first microphone Mic1 collects, with afirst voice recognition model 51. When the specific audio data isdetected, the first signal processing module 10 of the low-powerprocessing module 170 may transfer a wakeup command to the DOA decisionunit 40 of the audio codec 130.

The DOA decision unit 40 of the audio codec 130 may collect audio datafrom a microphone module MIC that the first signal processing module 10activates. In this operation, the DOA decision unit 40 of the audiocodec 130 may temporarily store (buffer) the collected audio data. Whenreceiving the wakeup command from the first signal processing module 10of the low-power processing module 170, the DOA decision unit 40 maydetermine a direction of arrival and generation of information, based onthe buffered audio data. For example, the DOA decision unit 40 maycalculate a parameter associated with weighting a plurality ofmicrophones Mic1 to MicN included in the microphone module MIC.

The DOA decision unit 40 of the audio codec 130 may transfer thecalculated weight parameter to the beamforming/noise suppression module50 of the audio codec 130 performing beamforming/noise suppression. TheDOA decision unit 40 of the audio codec 130 may use audio data, which afirst microphone Mic1 collects, and audio data, which second to N-thmicrophones Mic2 to MicN collect, to determine a direction of arrival.Audio data used to determine a direction of arrival may be audio dataused to detect specific audio data.

In the case where the low-power processing module 170 detects specificaudio data, the DOA decision unit 40 of the audio codec 130 may buffercollected audio data. When receiving a wakeup command from the firstsignal processing module 10 of the low-power processing module 170, theDOA decision unit 40 of the audio codec 130 may determine a direction ofarrival using the buffered audio data. If the wakeup command is notreceived after buffering specific audio data, the DOA decision unit 40of the audio codec 130 may delete the buffered data or may overwrite thebuffered data with subsequently received data. The DOA decision unit 40may calculate a weight parameter, based on audio data, and may transferthe weight parameter to the beamforming/noise suppression module 50.

The beamforming/noise suppression module 50 of the audio codec 130 mayapply weights to the microphones Mic1 to MicN according to a weightparameter received from the DOA decision unit 40 of the audio codec 130.The beamforming/noise suppression module 50 of the audio codec 130 maycompute audio data that undergoes noise suppression or is obtained byperforming beamforming of audio data, which the microphones Mic1 to MicNcollect, in a specific direction. The beamforming/noise suppressionmodule 50 may transfer audio data, to which at least one of beamformingor noise suppression is applied, to the second signal processing module20 of the audio codec 130. The beamforming/noise suppression module 50of the audio codec 130 may buffer audio data that the microphones Mic1to MicN collect. The beamforming/noise suppression module 50 of theaudio codec 130 may receive information of a specific audio data sectionfrom the DOA decision unit 40 of the audio codec 130. Upon weighting,the beamforming/noise suppression module 50 of the audio codec 130 mayexclude buffered audio data corresponding to the specific audio datasection. The beamforming/noise suppression module 50 of the audio codec130 may apply a weight to pieces of audio data corresponding to afunction execution command section.

The second signal processing module 20 of the audio codec 130 mayreceive audio data, which is obtained by applying beamforming or noisesuppression to audio data collected by the microphones Mic1 to MicN,from the beamforming/noise suppression module 50. The second signalprocessing module 20 of the audio codec 130 may execute voicerecognition of the received audio data. In this operation, the secondsignal processing module 20 of the audio codec 130 may execute voicerecognition, based on the second signal processing module 20.Additionally or generally, the second signal processing module 20 of theaudio codec 130 may preprocess the audio data thus received. The secondsignal processing module 20 of the audio codec 130 may transfer a voicerecognition result to the audio data processing module 180 to control toperform a function according to the recognition result. Alternatively,the audio codec 130 may control to perform a specific function

A first microphone Mic1 of the microphone module MIC may be coupled tothe first signal processing module 10 of the low-power processing module170, the DOA decision unit 40 of the audio codec 130, and thebeamforming/noise suppression module 50 of the audio codec 130. Secondto N-th microphones Mic2 to MicN may be coupled to the DOA decision unit40 of the audio codec 130 and the beamforming/noise suppression module50 of the audio codec 130. The microphone MIC may be activated accordingto a control of the first signal processing module 10. For example, thefirst microphone Mic1 may transfer first audio data corresponding tospecific audio data to the DOA decision unit 40 of the audio codec 130and the beamforming/noise suppression module 50 of the audio codec 130,respectively. The second to N-th microphones Mic2 to MicN may transferthe first audio data to the DOA decision unit 40 of the audio codec 130and the beamforming/noise suppression module 50 of the audio codec 130,respectively. Second audio data collected by the microphones Mic1 toMicN after the first audio data is collected may be transferred to thebeamforming/noise suppression module 50 of the audio codec 130 so as tobe converted into third audio data (parameter-processed audio data) towhich at least one of beamforming or noise suppression is applied. Thethird audio data thus converted may be transferred to the second signalprocessing module 20.

After calculating and transferring a weight parameter, the DOA decisionunit 40 of the audio codec 130 may buffer audio data collected untilreceiving a wakeup command from the first signal processing module 10 ofthe low-power processing module 170. According to an embodiment of thepresent disclosure, when receiving the wakeup command, the DOA decisionunit 40 of the audio codec 130 may determine a direction of arrivalassociated with buffered audio data corresponding to the wakeup command.For example, when “Hi Galaxy” is specific audio data, the DOA decisionunit 40 of the audio codec 130 may receive the wakeup command from thefirst signal processing module 10 at a point in time when “Hi Galaxy” iscollected, and may respond to the wakeup command to determine adirection of arrival based on audio data corresponding to “Hi Galaxy”.According to various embodiments of the present disclosure, the DOAdecision unit 40 of the audio codec 130 may determine a direction ofarrival associated with audio data collected in real time orperiodically and may transfer the decision result of the direction ofarrival to the beamforming/noise suppression module 50 of the audiocodec 130.

The DOA decision unit 40 is illustrated as being placed at the low-powerprocessing module 170 and as being placed at the audio codec 130 and thebeamforming/noise suppression module 50 is illustrated as being placedat the audio codec 130. However, a description on each device componentmay not be limited to the above-described embodiments. Positions ofdevice components may be modified according to a change in a designmanner. According to an embodiment of the present disclosure, the DOAdecision unit 40 and the beamforming/noise suppression module 50 may bedisposed at the audio data processing module 180. In FIGS. 5 and 6, thefirst signal processing module 10 is illustrated as being disposed atthe low-power processing module 170. However, the first signalprocessing module 10 can be placed at the audio codec 130. Furthermore,the second signal processing module 20 is illustrated as being disposedat the audio codec 130. However, the second signal processing module 20may be disposed at the audio data processing module 180.

As described above, the electronic device 100 according to variousembodiments of the present disclosure may include the first signalprocessing module, which activates the plurality of microphones Mic1 toMicN and detects specific audio data using first audio data, which afirst microphone Mic1 collects, from among audio data collected by theplurality of microphones Mic1 to MicN, a DOA decision unit 40, whichdetermines a direction of arrival using the first audio data, if thespecific audio data is detected, the beamforming/noise suppressionmodule 50, which applies at least one of beamforming or noisesuppression to collected audio data according to the direction ofarrival thus determined, and generates parameter-processed audio data,and the second signal processing module 20, which executes voicerecognition of the parameter-processed audio data.

According to various embodiments of the present disclosure, when thespecific audio data is detected, the first signal processing module 10may generate a wakeup command and may transfer the wakeup command to theDOA decision unit 40.

According to various embodiments of the present disclosure, the DOAdecision unit 40 may buffer first audio data which the plurality ofmicrophones Mic1 to MicN collects before receiving the wakeup command.

According to various embodiments of the present disclosure, the DOAdecision unit 40 may determine a sound obtaining direction using thepieces of the buffered first audio data, when the wakeup command isreceived.

According to various embodiments of the present disclosure, thebeamforming/noise suppression module 50 may buffer audio data which theplurality of microphones Mic1 to MicN collects before receiving thedirection of arrival.

According to various embodiments of the present disclosure, thebeamforming/noise suppression module 50 may apply at least one of thebeamforming or the noise suppression to second audio data, which theplurality of microphones Mic1 to MicN collects, excluding the firstaudio data.

According to various embodiments of the present disclosure, the secondsignal processing module 20 may control execution of a specific functioncorresponding to the voice-recognized result.

According to various embodiments of the present disclosure, theelectronic device 10 may include the low-power processing module 170 inwhich the firs signal processing module 10 and the DOA decision unit 40are disposed and the audio codec 130 or the audio data processing module180 in which the beamforming/noise suppression module 50 and the secondsignal processing module 20 are disposed.

According to various embodiments of the present disclosure, theelectronic device 100 may include a low-power processing module 170 inwhich the firs signal processing module 10 is disposed and an audiocodec 130 or an audio data processing module 180 in which the DOAdecision unit 40, the beamforming/noise suppression module 50, and thesecond signal processing module 20 are disposed.

The single channel signal processing module 11 may process first audiodata collected by a portion of a plurality of microphones. According toan embodiment of the present disclosure, at least a portion of thesingle channel signal processing module 11 may be implemented by a firstprocessor. The first processor may be a general-purpose processor (or acommunication processor (CP)) of an electronic device or may be anapplication processor (AP). The first processor may be separated fromthe general-purpose processor of the electronic device and may be adedicated processor for implementing an audio data processing function.A first voice recognition module 12 may execute voice recognition offirst audio data. The first voice recognition module 12 may executevoice recognition of first audio data. According to an embodiment of thepresent disclosure, at least a portion of the first voice recognitionmodule 12 may be implemented by a first processor.

A multi-channel signal processing module 30 may process second audiodata which a plurality of microphones collects. According to anembodiment of the present disclosure, at least a portion of themulti-channel signal processing module 30 may be implemented by a secondprocessor. The second processor may be a general-purpose processor (or acommunication processor (CP)) of an electronic device or may be anapplication processor (AP). The second processor may be separated fromthe general-purpose processor of the electronic device and may be adedicated processor for implementing an audio data processing function.

At least one of the preprocessing module 21 or the second voicerecognition module 22 of the second signal processing module 20 mayperform a function associated with voice recognition of second audiodata. According to an embodiment of the present disclosure, at least aportion of a preprocessing module 21 or a second voice recognitionmodule 22 may be implemented by a second processor.

According to various embodiments of the present disclosure, anelectronic device may include a plurality of microphones operativelycoupled to the electronic device and an audio data processing moduleimplemented by at least one processor. The audio data processing modulemay recognize a specified command, based on first audio data collectedusing a portion of the plurality of microphones and may execute afunction or an application corresponding to second audio data collectedusing the plurality of microphones, when the specified command isrecognized.

According to various embodiments of the present disclosure, the audiodata processing module may include a single channel signal processingmodule which receives an audio signal of at least one channelcorresponding to the portion of the plurality of microphones andgenerates the first audio data, based on a result of performing aspecified audio process associated with the audio signal of the at leastone channel, a first voice recognition module which recognizes thespecified command through voice recognition of the first audio data, amulti-channel signal processing module which receives a multi-channelaudio signal corresponding to each of the plurality of microphones andgenerates the second audio data, based on a result of performing aspecified audio process associated with the multi-channel audio signal,and a second voice recognition module which performs the function orapplication through voice recognition of the second audio data.

According to various embodiments of the present disclosure, the firstvoice recognition module may be implemented with a first processoperatively coupled to the portion of the plurality of microphones, andthe second voice recognition module may be implemented with a secondprocessor operatively coupled to the plurality of microphones.

According to various embodiments of the present disclosure, whenrecognizing the specified command, the first voice recognition modulemay activate at least one remaining microphone of the plurality ofmicrophones other than the portion of the plurality of microphones orthe multi-channel signal processing module.

According to various embodiments of the present disclosure, themulti-channel signal processing module may include at least one of asound source direction detecting unit which recognizes a sound sourcedirection of the multi-channel audio signal, a beam forming unit whichadjusts a parameter of the multi-channel audio signal so as to adjust areceiving gain of a specific direction, a noise suppression unit whichadjusts a parameter of the multi-channel audio signal so as to suppressreceiving of a sound source of a specific direction associated with anoise, or an echo cancellation unit which cancels an echo componentincluded in the multi-channel audio signal.

According to various embodiments of the present disclosure, the firstvoice recognition module may determine whether at least one of utterancecharacter information or speaker classification informationcorresponding to the specified command is included in the first audiodata.

According to various embodiments of the present disclosure, when thespecific audio data is detected, the single channel signal processingmodule may transfer a command, set to activate the multi-microphoneprocessing module, to the multi-microphone processing module.

According to various embodiments of the present disclosure, theelectronic device may include an audio codec in which at least one of asingle channel signal processing module or the multi-channel signalprocessing module is disposed and an audio data processing module inwhich the second voice recognition module is disposed.

According to various embodiments of the present disclosure, theelectronic device may include a low-power processing module in which thesingle channel signal processing module is disposed and an audio dataprocessing module in which the multi-channel signal processing moduleand the second voice recognition module are disposed.

According to various embodiments of the present disclosure, theelectronic device may include a low-power processing module in which thesingle channel signal processing module is disposed, an audio code inwhich the multi-channel signal processing module is disposed, and anaudio data processing module in which the second voice recognitionmodule is disposed.

FIG. 7 illustrates a microphone operating method according to anembodiment of the present disclosure.

Referring to FIG. 7, in operation 701, the audio data processing module180 of the electronic device 100 may wait or operate. With regard towaiting or operating, for example, the electronic device 100 maymaintain a sleep state, may output a waiting screen, or may execute aspecific sound source reproduction function.

In operation 703, the audio data processing module 180 may determinewhether an event associated with execution of a voice recognitionfunction exists. For example, when a specific input event is generated,the audio data processing module 180 may determine whether the inputevent is an input event associated with execution of the voicerecognition function. Alternatively, the audio data processing module180 may determine whether setting associated with execution of the voicerecognition function exists.

When the setting associated with execution of the voice recognitionfunction does not exist, a function associated with the input event orsetting is performed in operation 705. For example, the audio dataprocessing module 180 may change a previously executed function inresponse to the kind of the input event or may control to execute a newfunction. According to various embodiments of the present disclosure,operation 703 may be omitted if the voice recognition function is set tobe executed by default.

When an event associated with execution of the voice recognitionfunction is generated or the setting associated therewith exists inoperation 703, in operation 707, the audio data processing module 180may determine whether a power saving function of the voice recognitionfunction is set or whether an event or setting associated with executionof the power saving function exists. The power saving function may be avoice recognition function corresponding to a way of generating a wakeupcommand (or an active command, a command associated with moduleactivation associated with multi-microphone processing, and or the like)using one microphone (or at least one of a plurality of microphones) andactivating a plurality of microphones in generating the wakeup command.The audio data processing module 180 may provide a setting screenassociated with setting of the power saving function.

When the power saving function is set or an event associated with thepower saving function is generated, in operation 709, the electronicdevice 100 may control to activate a first microphone Mic1. In operation711, the electronic device 100 may collect first audio data using thefirst microphone Mic1 activated. In operation 713, the electronic device100 may determine whether the collected first audio data includesspecific audio data (or, whether similarity between the collected firstaudio data and a trained statistical model is within a constant errorrange). The specific audio data may be at least one of audio data forgenerating a wakeup command or the trained statistical model. Activatingof the first microphone Mic1 and generating of the wakeup command may beperformed by a first signal processing module 10 that is included in atleast one of the low-power processing module 170 and the audio codec130. The audio data processing module 180 may stop a process associatedwith the voice recognition function while the low-power processingmodule 170 and the an audio codec 130 employ the first signal processingmodule 10, thereby saving power needed to operate the audio dataprocessing module 180.

When the specific audio data is determined in operation 713 as beingincluded in the first audio data, in operation 715, the electronicdevice 100 may control to activate a plurality of microphones and toexecute the voice recognition function. For example, the electronicdevice 100 may perform a multi-microphone process associated with audiodata collected after the first audio data and may execute voicerecognition of the audio data undergoing the multi-microphone process.In this operation, the electronic device 100 may preprocess the audiodata undergoing the multi-microphone process. The electronic device mayexecute voice recognition of the preprocessed audio data using thesecond voice recognition model 53. Alternatively, the electronic device100 may transfer the preprocessed audio data to a voice recognitionserver device and may receive a voice recognition result therefrom.

When the voice recognition result is obtained, the electronic device 100may control to perform a specific function in response to the voicerecognition result. For example, the electronic device 100 may controlto perform a specific function in response to a voice recognition resultobtained as the voice recognition function. According to an embodimentof the present disclosure, based on the voice recognition result, theelectronic device 100 may enter a sleep state, may change a broadcastingreceiving channel, may control to reproduce a specific sound source, mayform a communication channel with another electronic device, may connectto a specific server device, and the like. Execution of a specificfunction may be changed or established according to a specific setting,a user setting, or the like.

When the power saving function is not set or an event is not generatedin operation 707, in operation 717, the electronic device 100 mayrecognize an event or setting associated with execution of the voicerecognition function as setting of a detail recognition function.Accordingly, in operation 719, the electronic device 100 may control toactivate a plurality of microphones Mic1 to MicN. In operation 721, theelectronic device 100 may collect audio data using the microphones Mic1to MicN. In operation 723, the electronic device 100 may determinewhether specific audio data is included in the collected audio data. Inthe case where the specific audio data is included in the collectedaudio data, the electronic device 100 determines a direction of arrivalin operation 725, and in operation 727, the electronic device 100 maycontrol to perform a beamforming/noise suppression-based voicerecognition function.

According to an embodiment of the present disclosure, at least one ofthe low-power processing module 170 or the audio codec 130 of theelectronic device 100 may use the first signal processing module 10which activates a plurality of microphones Mic1 to MicN and detectsspecific audio data with regard to generating and processing a wakeupcommand. Furthermore, a multi-channel signal processing module 30, whichperforms a multi-microphone processing function of the microphones Mic1to MicN, or the DOA decision unit 40 and the beamforming/noisesuppression module 50 may operate in an audio codec 130 or an audio dataprocessing module 180. The DOA decision unit 40 may operate in thelow-power processing module 170. A second signal processing module 20may perform a voice recognition function of audio data undergoing amulti-microphone process and may operate in one of the audio codec 130and the audio data processing module 180.

When determining a direction of arrival, the electronic device 100 mayuse specific audio data used to generate a wakeup command. Furthermore,when processing a parameter, the electronic device 100 may execute voicerecognition of audio data from which a section of specific audio data isexcluded. The electronic device 100 may continuously perform generationof a wakeup command and a voice recognition function using a pluralityof microphones Mic1 to MicN, thereby supporting the voice recognitionfunction seamlessly.

In operation 729, the electronic device 100 may determine whether anevent associated with a function end is generated. When an eventassociated with a function end is generated, the electronic device 100may terminate the voice recognition function, and the method may proceedto a previous operation of operation 701. When an event associated witha function end is not generated, the method proceeds to operation 703 oroperation 707, in which the electronic device 100 may repeat thecorresponding operations.

At least a portion of operations (e.g., operations 701 to 729) of themethod according to various embodiments of the present disclosure may beperformed sequentially, in parallel, or iteratively. Alternatively, aportion of the operations according to various embodiments of thepresent disclosure may be omitted, or a new operation may be addedthereto.

As described above, according to various embodiments of the presentdisclosure, a microphone operating method according to variousembodiments of the present disclosure may include determining a settingassociated with execution of a power saving function or a detailrecognition function or generation of an event associated therewith,activating a plurality of microphones based on audio data, which a firstmicrophone collects at execution of the power saving function, and amulti-microphone process and voice recognition of the collected audiodata (a power saving function based voice recognition operation), andcalculating and applying of a parameter according to a determination ofa sound obtaining direction and voice recognition, using audio datacollected by the plurality of microphones at execution of the detailrecognition function (a detail recognition function based voicerecognition operation).

According to various embodiments of the present disclosure, the powersaving function based voice recognition operation may includedetermining specific audio data is included in audio data collected bythe first microphone, activating a plurality of microphones Mic1 to MicNwhen the specific audio data is detected, performing a multi-microphoneprocess associated with audio data collected by the plurality ofmicrophones Mic1 to MicN, and executing voice recognition of the audiodata undergoing the multi-microphone process.

According to various embodiments of the present disclosure, thedetermining may include performing at least a portion of a plurality ofpreprocessing functions with respect to audio data collected by thefirst microphone Mic1 and executing voice recognition of the audio datacollected by the first microphone Mic1.

According to various embodiments of the present disclosure, thedetermining may include generating a wakeup command when the specificaudio data is detected and transferring the wakeup command to a moduleperforming the multi-microphone process.

According to various embodiments of the present disclosure, theactivating of the plurality of microphones may include allowing amodule, performing the multi-microphone process, to activate theplurality of microphones Mic1 to MicN in response to the wakeup command.

According to various embodiments of the present disclosure, theexecuting of voice recognition may include performing a plurality ofpreprocessing functions of the audio data undergoing themulti-microphone process and executing voice recognition of the audiodata undergoing the multi-microphone process, based on a second voicerecognition model 53.

According to various embodiments of the present disclosure, themulti-microphone process may include at least one of detecting adirection of arrival associated with audio data collected by theplurality of microphones, performing beamforming according to thedirection of arrival detected, suppressing a noise by suppressingobtaining of a sound of a specific direction with respect to the audiodata, and performing echo cancellation associated with the pieces ofaudio data.

According to various embodiments of the present disclosure, the detailrecognition function based voice recognition operation may includeactivating a plurality of microphones Mic1 to MicN, detecting specificaudio data using audio data collected by a first microphone Mic1, fromamong audio data collected by the plurality of microphones Mic1 to MicN,calculating the parameter using the audio data when the specific audiodata is detected, and executing voice recognition of theparameter-processed audio data.

According to various embodiments of the present disclosure, generating awakeup command when the specific audio data is detected and transferringthe wakeup command to a module determining the sound obtaining directionmay be further included.

According to various embodiments of the present disclosure, thecalculating of a parameter may include buffering audio data which theplurality of microphones collects before receiving the wakeup command,and calculating the parameter according to a determination of a soundobtaining direction using the buffered audio data, when the wakeupcommand is received.

According to various embodiments of the present disclosure, the applyingof the parameter may include audio data which the plurality ofmicrophones collects before determining the sound obtaining direction,and applying at least one of the beamforming or the noise suppression toaudio data, from which audio data used to detect the specific audio dataare excluded, from among audio data collected by the plurality ofmicrophones Mic1 to MicN.

According to various embodiments of the present disclosure, executing aspecific function corresponding to the voice-recognized result may befurther included.

FIG. 8 illustrates a screen interface of an electronic device accordingto various embodiments of the present disclosure.

Referring to FIG. 8, when an event associated with setting of a voicerecognition function is generated, a display 140 may output a powersaving function selecting icon 141 and a detail recognition functionselecting icon 143 as illustrated in FIG. 8. When an event associatedwith activation of the voice recognition function is generated, theaudio data processing module 180 may control output a setting screen asillustrated in FIG. 8. The audio data processing module 180 may providea menu or icon including a voice recognition function activating orinactivating item. According to various embodiments of the presentdisclosure, when the voice recognition function is set to be performedby default, the menu or icon including the function activating orinactivating item may not be provided (or may be omitted).

When the power saving function selecting icon 141 is selected, the firstsignal processing module 10 included in either the low-power processingmodule 170 or the audio codec 130 of the electronic device 100 mayactivate a first microphone Mic1 to detect specific audio data. When thespecific audio data is detected, the first signal processing module 10may generate a wakeup command and may transfer the wakeup command to amulti-channel signal processing module 30. Alternatively, the firstsignal processing module 10 may transfer the wakeup command to the DOAdecision unit 40. The multi-channel signal processing module 30 may bedisposed at the audio codec 130 or the audio data processing module 180.

When receiving the wakeup command, the multi-channel signal processingmodule 30 may activate the plurality of microphones Mic1 to MicN and mayperform a multi-microphone process associated with the collected audiodata. Alternatively, the DOA decision unit 40 may determine a directionof arrival associated with the collected audio data and may transfer thedirection of arrival to the beamforming/noise suppression module 50. Thebeamforming/noise suppression module 50 may collect audio data to whichbeamforming or noise suppression is applied according to the directionof arrival thus determined. The DOA decision unit 40 may be disposed atthe low-power processing module 170 or the audio codec 130. Thebeamforming/noise suppression module 50 may be disposed at the audiocodec 130 or the audio data processing module 180. As described above,the low-power processing module 170, which is driven by relatively lowpower, may process audio data of which the load or computation isrelatively less, and the audio codec 130, which is driven by relativelyhigh power, or the audio data processing module 180 may process audiodata of which the load is relatively more.

Audio data undergoing a multi-microphone process may be provided to thesecond signal processing module 20 that is arranged at the audio codec130 or the audio data processing module 180. The second signalprocessing module 20 may preprocess the received audio data undergoingthe multi-microphone process and may execute voice recognition using thesecond voice recognition model 53 or a voice recognition server device.The second signal processing module 20 may control to perform a specificfunction according to a voice recognition result. Alternatively, adevice component including the second signal processing module 20 maycontrol to perform a specific function according to a voice recognitionresult in response to setting information.

When the detail recognition function selecting icon 143 is selected, theelectronic device 100 may activate a plurality of microphones Mic1 toMicN. The electronic device 100 may detect specific audio data using afirst microphone Mic1 of the plurality of microphones Mic1 to MicN. Whenthe specific audio data is detected, the electronic device 100 maydetermine a direction of arrival using audio data that is used to detectaudio data. The electronic device 100 may apply a beamforming or noisesuppression function to audio data, continuously collected by themicrophones Mic1 to MicN, except the specific audio data. The electronicdevice 100 may preprocess the beam-formed or noise-suppressed audio dataand may execute regulation voltage of the preprocessed audio data.

The electronic device 100 may manage the power saving function selectingicon 141 or the detail recognition function selection icon 143, forexample, in a toggle manner. For example, when the power saving functionselecting icon 141 is selected, the detail recognition function and thedetail recognition function selection icon 143 may be automaticallyinactivated according to a control of the electronic device 100.Furthermore, in the case where the detail recognition function selectionicon 143 is selected, the power saving function and the power savingfunction selecting icon 141 may be automatically inactivated accordingto a control of the electronic device 100. According to variousembodiments of the present disclosure, the electronic device 100 mayprovide a selection item for inactivating or activating the voicerecognition function.

As described above, an electronic device 100 and its operating methodaccording to various embodiments of the present disclosure may use twoor more processors to support such that it is possible to wait for avoice input while maintaining low power even at a waiting state.Furthermore, the electronic device 100 and the operating methodaccording to an embodiment of the present disclosure may support toobtain high-quality sound using a multi-microphone while waiting for avoice input at a low-power state. In addition, the electronic device 100and the operating method according to an embodiment of the presentdisclosure may process to receive a wakeup command and a functionexecution command seamlessly using at least one processor. According tovarious embodiments of the present disclosure, it is possible to receivenatural language voice while waiting at a low-power state, therebyimproving convenience of a user.

As described above, a microphone operating method according to anembodiment of the present disclosure may include collecting first audiodata using a portion of a plurality of microphones operatively coupledto an electronic device, recognizing a specified command, based on thefirst audio data, and executing a function or an application,corresponding to second audio data collected using all the plurality ofmicrophones, based on recognition of the specified command.

According to various embodiments of the present disclosure, themicrophone operating method may further include at least one of whereincollecting the first audio data comprises performing a single channelsignal processing operation of generating the first audio data, based ona result of performing a specified audio process associated with anaudio signal of at least one channel corresponding to a portion of theplurality of microphones; wherein recognizing the specified commandcomprises performing a first voice recognition operation of recognizingthe specified command through voice recognition of the first audio data;wherein executing the function or application comprises performing amulti-channel signal processing operation of generating the second audiodata, based on a result of performing a specified audio processassociated with a multi-channel audio signal corresponding to each ofthe plurality of microphones; or wherein executing the function orapplication comprises performing a second voice recognition operation ofperforming the function or the application through voice recognition ofthe second audio data.

According to various embodiments of the present disclosure, themicrophone operating method may further include at least one ofperforming the first voice recognition operation by a first processoroperatively coupled to the at least one microphone or performing thesecond voice recognition by a second processor operatively coupled tothe plurality of microphones.

According to various embodiments of the present disclosure, themicrophone operating method may further include at least one ofactivating, when the specified command is recognized, remainingmicrophones of the plurality of microphones other than the portion ofthe plurality of microphones or processing multi-channel signal when thespecified command is recognized.

According to various embodiments of the present disclosure, themicrophone operating method may further include at least one ofdetermining a sound source direction of the multi-channel audio signal,based on positions where microphones corresponding to the multi-channelaudio signal are disposed, adjusting a parameter of the multi-channelaudio signal so as to tune an input gain of a specific direction,adjusting a parameter of the multi-channel audio signal so as tosuppress receiving of sound source of a specific direction associatedwith a noise, or cancelling an echo component included in themulti-channel audio signal.

According to various embodiments of the present disclosure, the firstvoice recognition operation may include determining whether at least oneof utterance character information or speaker classification informationcorresponding to the specified command is included the first audio data.

According to various embodiments of the present disclosure, themicrophone operating method may further include transferring a command,set to process the multi-channel audio signal, to a multi-channelprocessing module, when the specific audio data is detected.

According to various embodiments of the present disclosure, themicrophone operating method may further include at least one of settingat least one of the single channel signal processing operation or themulti-channel signal processing operation to an audio codec or settingthe second voice recognition operation to an audio data processingmodule.

According to various embodiments of the present disclosure, themicrophone operating method may further include at least one of settingthe single channel signal processing operation to a low-power processingmodule or setting the multi-channel signal processing operation and thesecond voice recognition operation to an audio data processing module.

According to various embodiments of the present disclosure, themicrophone operating method may further include at least one of settingthe single channel signal processing operation to a low-power processingmodule, setting the multi-channel signal processing operation to anaudio codec, or setting the second voice recognition operation to anaudio data processing module.

FIG. 9 illustrates a hardware configuration of an electronic deviceaccording to various embodiments of the present disclosure.

Referring to FIG. 9, an electronic device 900 may include a part or allof components of the electronic device 100 shown in FIG. 1. Theelectronic device 900 may include one or more application processors(AP) 910, a communication module 920 (e.g., the communication interface110), a subscriber identification module (SIM) card 924, a memory 930(e.g., the memory 150), a sensor module 940, an input device 950 (e.g.,the input/output interface 120), a display(s) 960 (e.g., the display140), an interface 970, an audio module 980 (e.g., the input/outputinterface 120), a camera module 991, a power management module 995, abattery 996, an indicator 997, or a motor 998.

The AP 910 may drive an operating system (OS) or an application tocontrol a plurality of hardware or software components connected to theAP 910 and may process and compute a variety of data includingmultimedia data. The AP 910 may be implemented with a System on Chip(SoC), for example. According to an embodiment of the presentdisclosure, the AP 910 may further include a graphic processing unit(GPU) (not illustrated).

The communication module 920 (e.g., the communication interface 110) maytransmit and receive data when there are conveyed communications betweenother electronic devices connected with the electronic device 900through a network. According to an embodiment of the present disclosure,the communication module 920 may include a cellular module 921, a Wi-Fimodule 923, a Bluetooth (BT) module 925, a global positioning system(GPS) module 927, a near field communication (NFC) module 928, and aradio frequency (RF) module 929.

The cellular module 921 may provide voice communication, videocommunication, a character service, an Internet service, and the likethrough a communication network (e.g., LTE, LTE-A, CDMA, WCDMA, UMTS,WiBro, GSM, or the like). The cellular module 921 may performidentification and authentication of an electronic device within acommunication network using, for example, a subscriber identificationmodule (e.g., the SIM card 924). According to an embodiment of thepresent disclosure, the cellular module 921 may perform at least aportion of functions that the AP 910 provides. For example, the cellularmodule 921 may perform at least a portion of a multimedia controlfunction.

According to an embodiment of the present disclosure, the cellularmodule 921 may include a communication processor (CP). Furthermore, thecellular module 921 may be implemented with, for example, a SoC.Although components such as the cellular module 921 (e.g., acommunication processor), the memory 930, the power management module995, and the like are illustrated as being components independent of theAP 910, the AP 910 according to an embodiment of the present disclosuremay be implemented to include at least a portion (e.g., a cellularmodule 921) of the above components.

According to an embodiment of the present disclosure, the AP 910 or thecellular module 921 (e.g., a communication processor) may load andprocess an instruction or data received from nonvolatile memoriesrespectively connected thereto or from at least one of other elements atthe nonvolatile memory. The AP 910 or the cellular module 921 may storedata received from at least one of other elements or generated by atleast one of other elements at a nonvolatile memory.

Each of the Wi-Fi module 923, the BT module 925, the GPS module 927, andthe NFC module 928 may include a processor for processing data exchangedthrough a corresponding module, for example. In FIG. 9, the cellularmodule 921, the Wi-Fi module 923, the BT module 925, the GPS module 927,and the NFC module 928 may be illustrated as being separate blocks,respectively. According to an embodiment of the present disclosure, atleast a portion (e.g., two or more components) of the cellular module921, the Wi-Fi module 923, the BT module 925, the GPS module 927, andthe NFC module 928 may be included within one Integrated Circuit (IC) oran IC package. For example, at least a portion (e.g., a communicationprocessor corresponding to the cellular module 921 and a Wi-Fi processorcorresponding to the Wi-Fi module 923) of communication processorscorresponding to the cellular module 921, the Wi-Fi module 923, the BTmodule 925, the GPS module 927, and the NFC module 928 may beimplemented with one SoC.

The RF module 929 may transmit and receive data, for example, an RFsignal. Although not illustrated, the RF module 929 may include atransceiver, a power amplifier module (PAM), a frequency filter, or lownoise amplifier (LNA). Furthermore, the RF module 929 may furtherinclude a part for transmitting and receiving an electromagnetic wave ina space in wireless communication: a conductor or a conducting wire. InFIG. 9, the cellular module 921, the Wi-Fi module 923, the BT module925, the GPS module 927, and the NFC module 928 may be illustrated assharing one RF module 929, but according to an embodiment of the presentdisclosure, at least one of the cellular module 921, the Wi-Fi module923, the BT module 925, the GPS module 927, or the NFC module 928 maytransmit and receive an RF signal through a separate RF module.

The SIM card 924 may be a card that includes a subscriber identificationmodule and may be inserted to a slot formed at a specific position ofthe electronic device 900. The SIM card 924 may include unique identifyinformation (e.g., integrated circuit card identifier (ICCID)) orsubscriber information (e.g., integrated mobile subscriber identity(IMSI)).

The memory 930 (e.g., the memory 130) may include an embedded memory 932or an external memory 934. For example, the embedded memory 932 mayinclude at least one of a volatile memory (e.g., dynamic RANI (DRAM),static RAM (SRAM), synchronous dynamic RAM (SDRAM), etc.), or anonvolatile memory (e.g., a dynamic random access memory (DRAM), astatic RAM (SRAM), or a synchronous DRAM (SDRAM)) and a nonvolatilememory (e.g., a one-time programmable read only memory (OTPROM), aprogrammable ROM (PROM), an erasable and programmable ROM (EPROM), anelectrically erasable and programmable ROM (EEPROM), a mask ROM, a flashROM, a NAND flash memory, or a NOR flash memory).

According to an embodiment of the present disclosure, the embeddedmemory 932 may be a solid state drive (SSD). The external memory 934 mayinclude a flash drive, for example, compact flash (CF), secure digital(SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD),extreme digital (xD) or a memory stick. The external memory 934 may befunctionally connected with the electronic device 900 through variousinterfaces. According to an embodiment of the present disclosure, theelectronic device 900 may further include a storage device (or storagemedium) such as a hard disk drive.

The sensor module 940 may measure a physical quantity or may detect anoperation state of the electronic device 900. The sensor module 940 mayconvert the measured or detected information to an electric signal. Thesensor module 940 may include at least one of a gesture sensor 940A, agyro sensor 940B, a pressure sensor 940C, a magnetic sensor 940D, anacceleration sensor 940E, a grip sensor 940F, a proximity sensor 940G, acolor sensor 940H (e.g., red, green, blue (RGB) sensor), a living bodysensor 9401, a temperature/humidity sensor 940J, an illuminance sensor940K, or an ultraviolet (UV) sensor 940M. Additionally or generally,although not illustrated, the sensor module 940 may further include, forexample, an E-nose sensor, an electromyography sensor (EMG) sensor, anelectroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, aphotoplethysmographic (PPG) sensor, an infrared (IR) sensor, an irissensor, a fingerprint sensor, and the like. The sensor module 940 mayfurther include a control circuit for controlling at least one or moresensors included therein.

The input device 950 may include a touch panel 952, a (digital) pensensor 954, a key 956, or an ultrasonic input unit 958. The touch panel952 may recognize a touch input using at least one of capacitive,resistive, infrared and ultrasonic detecting methods. The touch panel952 may further include a control circuit. In the case of using thecapacitive detecting method, a physical contact or proximity recognitionis possible. The touch panel 952 may further include a tactile layer. Inthis case, the touch panel 952 may provide a tactile reaction to a user.The touch panel 952 may generate a touch event associated with executionof a specific function using position associated information.

The (digital) pen sensor 954 may be implemented in a similar or samemanner as the method of receiving a touch input of a user or may beimplemented using an additional sheet for recognition. The key 956 mayinclude, for example, a physical button, an optical key, or a keypad.The ultrasonic input device 958, which is an input device for generatingan ultrasonic signal, may enable the electronic device 900 to detect asound wave through a microphone (e.g., a microphone module MIC,collecting first audio data using a portion of a plurality ofmicrophones and collecting second audio data using the plurality ofmicrophones) so as to identify data, wherein the ultrasonic input device958 is capable of wireless recognition. According to an embodiment thepresent disclosure, the electronic device 900 may use the communicationmodule 920 so as to receive a user input from an external device (e.g.,a computer or server) connected to the communication module 920.

The display 960 (e.g., the display 140) may include a panel 962, ahologram device 964, or a projector 966. The panel 962 may be a LCD oran active-matrix organic light-emitting diode (AMOLED). The panel 962may be, for example, flexible, transparent or wearable. The panel 962and the touch panel 952 may be integrated into a single module. Thehologram device 964 may display a stereoscopic image in a space using alight interference phenomenon. The projector 966 may project light ontoa screen so as to display an image. The screen may be arranged in theinside or the outside of the electronic device 200. According to variousembodiments of the present disclosure, the display 960 may furtherinclude a control circuit for controlling the panel 962, the hologramdevice 964, or the projector 966.

The interface 970 may include, for example, a high-definition multimediainterface (HDMI) 972, a universal serial bus (USB) 974, an opticalinterface 976, or a D-sub (D-subminiature) 978. Additionally orgenerally, the interface 970 may include, for example, a mobile highdefinition link (MHL) interface, a SD card/multi-media card (MMC)interface, or an infrared data association (IrDA) standard interface.

The audio module 980 may convert a sound and an electric signal in dualdirections. At least a portion of the audio module 980 may be includedin for example, an input/output interface 140 illustrated in FIG. 1. Theaudio module 980 may process, for example, sound information that isinput or output through a speaker 982, a receiver 984, an earphone 986,a microphone 988, or the like.

According to various embodiments of the present disclosure, themicrophone 988 included in the audio module 980 may include a pluralityof microphones. A portion (one microphone or microphones of which thenumber is less than that of the whole microphones) of the plurality ofmicrophones may be used to collect first audio data. In the case where asignal or information corresponding to a specified command is includedin the first audio data, all or a portion of the plurality ofmicrophones may be used to collect second audio data. The first audiodata and the second audio data may be included in utterance informationcontinuously uttered. Alternatively, the first audio data and the secondaudio data may be divided into words or meaningful words or by unitssuch as sentence, respiration, and the like.

According to various embodiments of the present disclosure, when afunction associated with the second audio data is performed, there maybe used at least one of results of executing voice recognition of thefirst audio data or the second audio data. For example, a function orapplication may be executed that is mapped onto at least one of firstinformation being a voice recognition result of the first audio data orsecond information being a voice recognition result of the second audiodata. Alternatively, an application that is running may be controlledaccording to the first information or the second information. Withregard to this, the electronic device 100 may manage a function tablemapped onto at least one of the first information or the secondinformation.

The camera module 991 for shooting a still image or a video may includeat least one image sensor (e.g., a front sensor or a rear sensor), alens (not illustrated), an image signal processor (ISP, notillustrated), or a flash (e.g., an LED or a xenon lamp, notillustrated).

The power management module 995 may manage power of the electronicdevice 900. Although not illustrated, the power management module 995may include, for example, a power management integrated circuit (PMIC) acharger IC, or a battery or fuel gauge.

The PMIC may be mounted on an integrated circuit or a SoC semiconductor.A charging method may be classified into a wired charging method and awireless charging method. The charger IC may charge a battery, and mayprevent an overvoltage or an overcurrent from being introduced from acharger. According to an embodiment of the present disclosure, thecharger IC may include a charger IC for at least one of the wiredcharging method and the wireless charging method. The wireless chargingmethod may include, for example, a magnetic resonance method, a magneticinduction method, or an electromagnetic method, and may include anadditional circuit, for example, a coil loop, a resonant circuit, or arectifier, and the like.

The battery gauge may measure, for example, a remaining capacity of thebattery 996 and a voltage, current or temperature thereof while thebattery is charged. The battery 996 may store or generate electricity,and may supply power to the electronic device 900 using the stored orgenerated electricity. The battery 996 may include, for example, arechargeable battery or a solar battery.

The indicator 997 may display a specific state of the electronic device900 or a part thereof (e.g., the AP 910), such as a booting state, amessage state, a charging state, and the like. The motor 998 may convertan electrical signal into a mechanical vibration. Although notillustrated, a processing device (e.g., a GPU) for supporting a mobileTV may be included in the electronic device 900. The processing devicefor supporting a mobile TV may process media data according to thestandards of DMB, digital video broadcasting (DVB) or media flow.

Each of the above-mentioned elements of the electronic device accordingto various embodiments of the present disclosure may be configured withone or more components, and the names of the elements may be changedaccording to the type of the electronic device. The electronic deviceaccording to various embodiments of the present disclosure may includeat least one of the above-mentioned elements, and some elements may beomitted or other additional elements may be added. Furthermore, some ofthe elements of the electronic device according to various embodimentsof the present disclosure may be combined with each other so as to formone entity, so that the functions of the elements may be performed inthe same manner as before the combination.

The term “module” used herein may represent, for example, a unitincluding one or more combinations of hardware, software and firmware.The term “module” may be interchangeably used with the terms “unit”,“logic”, “logical block”, “component” and “circuit”. The “module” may bea minimum unit of an integrated component or may be a part thereof. The“module” may be a minimum unit for performing one or more functions or apart thereof. The “module” may be implemented mechanically orelectronically. For example, the “module” according to variousembodiments of the present disclosure may include at least one of anapplication-specific IC (ASIC) chip, a field-programmable gate array(FPGA), and a programmable-logic device for performing some operations,which are known or will be developed.

According to various embodiments of the present disclosure, at least aportion of an apparatus (e.g., modules or functions thereof) or a method(e.g., operations) according to various embodiments of the presentdisclosure may be implemented, for example, by instructions stored in acomputer-readable storage media in the form of a programmable module.The instruction, when executed by one or more processors (e.g., theprocessor 910), may perform a function corresponding to the instruction.The computer-readable storage media may be, for example, the memory 930.At least a portion of the programming module may be implemented (e.g.,executed), for example, by the processor 910. At least a portion of theprogramming module may include the following for performing one or morefunctions: a module, a program, a routine, sets of instructions, aprocess, or the like.

A computer-readable recording medium may include a hard disk, a magneticmedia such as a floppy disk and a magnetic tape, an optical media suchas Compact Disc Read Only Memory (CD-ROM) and a DVD, a magneto-opticalmedia such as a floptical disk, and the following hardware devicesspecifically configured to store and perform a program instruction(e.g., a programming module): Read Only Memory (ROM), Random AccessMemory (RAM), and a flash memory. Also, a program instruction mayinclude not only a mechanical code such as things generated by acompiler but also a high-level language code executable on a computerusing an interpreter. The above hardware unit may be configured tooperate via one or more software modules for performing an operation ofthe present disclosure, and vice versa.

A module or a programming module according to an embodiment of thepresent disclosure may include at least one of the above elements, or aportion of the above elements may be omitted, or additional otherelements may be further included. Operations performed by a module, aprogramming module, or other elements according to an embodiment of thepresent disclosure may be executed sequentially, in parallel,repeatedly, or in a heuristic method. Also, a portion of operations maybe executed in different sequences, omitted, or other operations may beadded.

According to a microphone operating method and an electronic devicesupporting the same, various embodiments of the present disclosure mayimprove voice recognition performance.

Furthermore, various embodiments of the present disclosure may reduceenergy by using power efficiently.

While the present disclosure has been shown and described with referenceto various embodiments thereof, it will be understood by those skilledin the art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the present disclosure asdefined by the appended claims and their equivalents.

What is claimed is:
 1. An electronic device comprising: a plurality ofmicrophones; and one or more processors operatively coupled to pluralityof microphones, wherein the one or more processors, upon execution ofinstructions, are configured to: recognize a specified command byprocessing first audio data collected through a first set of theplurality of microphones, perform a specified audio process associatedwith a multi-channel audio signal corresponding to a second set of theplurality of microphones, the second set comprising at least twomicrophones, a number of microphones in the second set being larger thana number of microphones in the first set, and execute a function or anapplication corresponding to second audio data collected through thesecond set after the specified command is recognized, wherein at leastone of the one or more processors are further configured to identify asound source direction of the multi-channel audio signal for audiobeam-forming based on positions of microphones, and wherein the firstaudio data comprises at least one of utterance character information, orspeaker classification information corresponding to the specifiedcommand.
 2. The electronic device of claim 1, wherein the one or moreprocessors are further configured to: perform voice recognition of thesecond audio data, and execute the function or the applicationcorresponding to the voice recognition result.
 3. The electronic deviceof claim 2, wherein a first processor of the one or more processors isconfigured to operatively couple to a portion of the plurality ofmicrophones, and wherein a second processor of the one or moreprocessors is configured to operatively couple to the plurality ofmicrophones.
 4. The electronic device of claim 3, wherein the one ormore processors are further configured to: activate at least one of aremaining microphone of the plurality of microphones other than theportion of the plurality of microphones, or recognize the specifiedcommand using multi-channel signal processing.
 5. The electronic deviceof claim 2, further comprising at least one of: a noise suppressorconfigured to adjust a parameter of the multi-channel audio signal so asto suppress receiving of a sound source of a specific directionassociated with a noise; or an echo canceler configured to cancel anecho component included in the multi-channel audio signal.
 6. Theelectronic device of claim 1, wherein the specified audio processassociated with the multi-channel audio signal performed by an audiocodec of the one or more processors.
 7. The electronic device of claim1, wherein the specified audio process associated with the multi-channelaudio signal performed by an application processor of the one or moreprocessors.
 8. A microphone operating method for a mobile electronicdevice, the microphone operating method comprising: collecting firstaudio data through a first set of a plurality of microphones operativelycoupled to the mobile electronic device; recognizing a specified commandby processing the first audio data; performing a specified audio processassociated with a multi-channel audio signal corresponding to a secondset of the plurality of microphones, the second set comprising at leasttwo microphones, a number of microphones of the second set being largerthan a number of microphones of the first set; executing, by one or moreprocessors, a function or an application, corresponding to second audiodata through the second set of the plurality of microphones after thespecified command is recognized; and identifying, by the one orprocessors, a sound source direction of the multi-channel audio signalfor audio beam-forming based on positions of microphones, wherein thefirst audio data comprises at least one of utterance characterinformation, or speaker classification information corresponding to thespecified command.
 9. The microphone operating method of claim 8,wherein the executing of the function or the application comprises:performing a second voice recognition operation of performing thefunction or the application, through voice recognition of the secondaudio data.
 10. The microphone operating method of claim 9, furthercomprising at least one of: adjusting, by the one or processors, aparameter of the multi-channel audio signal so as to adjust an inputgain of microphones of a specific direction; performing the voicerecognition operation by a first processor operatively coupled to aportion of the plurality of microphones; or performing the second voicerecognition operation by a second processor operatively coupled to allof the plurality of microphones.
 11. The microphone operating method ofclaim 9, further comprising at least one of: activating at least one ofa remaining microphone of the plurality of microphones other than aportion of the plurality of microphones according to recognition of thespecified command; or processing the multi-channel audio signalaccording to the recognition of the specified command.
 12. Themicrophone operating method of claim 9, further comprising at least oneof: adjusting a parameter of the multi-channel audio signal so as tosuppress receiving of a sound source of a specific direction associatedwith a noise; or cancelling an echo component included in themulti-channel audio signal.
 13. The microphone operating method of claim9, further comprising: transferring a command, set to process themulti-channel audio signal, to a multi-channel processing moduleaccording to the recognition of the specified command or specific audiodata.
 14. The microphone operating method of claim 9, further comprisingat least one of: setting at least one of a single channel signalprocessing operation or a multi-channel signal processing operation toan audio codec; or setting the second voice recognition operation to anaudio data processing module.
 15. The microphone operating method ofclaim 8, wherein the specified audio process associated with themulti-channel audio signal performed by an audio codec of the one ormore processors.
 16. The microphone operating method of claim 8, whereinthe specified audio process associated with the multi-channel audiosignal performed by an application processor of the one or moreprocessors.
 17. An electronic device comprising: a plurality ofmicrophones operatively coupled to the electronic device; and one ormore processors including an audio codec and a low-power processingmodule, upon execution of instructions, configured to: recognize aspecified command by processing first audio data collected through afirst set of the plurality of microphones, perform a specified audioprocess associated with a multi-channel audio signal corresponding to asecond set of the plurality of microphones, the second set comprising atleast two microphones, a number of microphones in the second set beinglarger than a number of microphones in the first set, and execute afunction or an application corresponding to second audio data collectedthrough the second set after the specified command is recognized,wherein at least one of the one or more processors are furtherconfigured to identify a sound source direction of the multi-channelaudio signal for audio beam-forming based on positions of microphones,and wherein the first audio data comprises at least one of utterancecharacter information, or speaker classification informationcorresponding to the specified command.
 18. The electronic device ofclaim 17, wherein the specified audio process associated with themulti-channel audio signal performed by the audio codec of the one ormore processors.
 19. The electronic device of claim 17, wherein thespecified audio process associated with the multi-channel audio signalperformed by an application processor of the one or more processors.